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The  authors  developed  a  short  course  of  introductory  qualitative  research  methods  to  help 
the  lessons  learned  organizations  in  the  police  community  improve  their  data  collection  tech¬ 
niques.  This  document  provides  an  annotated  version  of  the  course  material.  It  should  be  of 
interest  to  research  professionals  interested  in  qualitative  research  methods. 

This  research  was  conducted  within  the  Intelligence  Policy  Center  of  the  RAND  National 
Defense  Research  Institute,  a  federally  funded  research  and  development  center  sponsored  by 
the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Unified  Combatant  Commands,  the 
Department  of  the  Navy,  the  Marine  Corps,  the  defense  agencies,  and  the  defense  Intelligence 
Community.  The  authors  of  this  work  are  Margaret  Harrell  and  Melissa  Bradley.  Comments 
are  welcome  and  may  be  addressed  to  Margaret_Harrell@rand.org  and  Melissa_Bradley@ 
rand.org. 

For  more  information  on  RAND’s  Intelligence  Policy  Center,  contact  the  Director,  John 
Parachini.  He  can  be  reached  by  email  at  John_Parachini@rand.org;  by  phone  at  703-413- 
1100  extension  5579;  or  by  mail  at  the  RAND  Corporation,  1200  S.  Hayes  Street,  Arlington, 
Virginia,  22202.  More  information  about  RAND  is  available  at  www.rand.org. 
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Data  Collection  Methods: 
Semi-Structured  Interviews 
and  Focus  Groups 

Margaret  C.  Harrell 
Melissa  A.  Bradley 


This  course  provides  an  overview  of  two  types  of  qualitative  data  collection  methodologies: 
semi-structured  interviews  and  focus  groups.  These  techniques  are  commonly  used  in  policy 
research  and  are  applicable  to  many  research  questions. 

The  course  does  not  aim  to  cover  all  the  aspects  of  interviewing  and  focus  groups,  but 
rather  it  is  designed  to  provide  an  overview  of  these  methods,  as  well  as  some  practical  tools 
that  will  allow  readers  to  improve  their  qualitative  data  collection. 

We  acknowledge  that  different  disciplines  often  speak  about  these  research  methods  in 
different  terms.  For  example,  while  some  disciplines  and  professions  may  use  the  term  “active 
data  collection,”  these  materials  use  the  terms  more  traditionally  associated  with  qualitative 
research. 


1 


2  Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups 


Using  the  Right  Techniques 


•  Ensures  that  data  are  collected  in  a  scientific  and 
standardized  manner 

*  Results  in  high-quality  research  and  thus, 
credible  findings 


RAND 


Primary  data  collection  is  an  important  piece  of  many  research  projects.  Using  proper  tech¬ 
niques  ensures  that  qualitative  data  are  collected  in  a  scientific  and  consistent  manner.' 

Improving  data  collection  techniques  will  enhance  the  accuracy,  validity,  and  reliability 
of  research  findings.  Ultimately,  using  these  methods  will  help  to  achieve  the  goal  of  carrying 
out  high-quality  research  with  credible  findings. ^ 


'  Throughout  this  work,  we  have  noted  resources  that  were  especially  helpful  or  were  the  sources  of  specific  examples. 
However,  much  of  this  information  exists  in  the  general  research  community  as  methodological  wisdom,  and  thus  our 
material  may  also  have  been  published  in  other  resources. 

Adapted  from  RAND  Survey  Research  Group  (SRG)  internal  training  materials  and  best  practices  documentation. 
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The  Research  Frame  Influences  the  Results 

*  Data  sources  can  be  framed  differently 

-  Informants  define  and  classify  themselves  and  their  culture; 
researcher  avoids  culture-bound  questions 

-  Subjects  used  to  test  hypotheses 

-  Respondents  respond  in  researcher’s  language  and  concepts 

-  Actors  are  observed,  in  researcher’s  language 

*  Be  conscious  of  disciplinary  differences 

*  Substantive  expertise  will  inform  your  findings,  but 
may  limit  your  queries 
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There  are  differences  between  the  semantics  of  traditional  academic  disciplines  involved  in 
qualitative  research  methods.  In  some  instances,  however,  the  difference  in  the  approach  to 
the  source  of  the  data  is  more  than  a  just  a  wording  difference.  For  example,  consider  the  dif¬ 
ferences  between  informants,  subjects,  respondents,  and  actors.^  Bernard  (2000)  notes  that 
anthropology  generally  uses  informants,  whereas  sociology  depends  upon  respondents.  That 
distinction  is  actually  very  important  to  the  difference  between  those  disciplines. 

Levy  and  Hollan  (1998)  distinguish  between  informants,  who  describe  their  culture,  and 
respondents,  who  discuss  characteristics,  beliefs,  experiences,  and  behavior.  Tfiat  is  a  partial 
distinction,  made  more  useful  when  one  adds  Spradley’s  (1979)  distinction  between  infor¬ 
mants  with  subjects,  respondents,  and  actors. 

Under  Spradley’s  terminology,  subjects  are  used  to  test  hypotheses.  The  researcher  believes 
he  knows  what  he  will  find,  posits  a  hypothesis,  and  goes  to  confirm  or  deny  the  hypothesis.  In 
this  instance,  the  researcher  is  seeking  the  scientific  theory  that  explains  the  observed  behav¬ 
ior  of  the  subjects.  Spradley  uses  an  example  of  checking  the  alcoholic  status  of  male  identical 
twins.  In  this  research,  the  scientists  were  exploring  whether  environmental  conditioning  was  a 
significant  factor  in  the  development  of  alcoholism.  The  important  thing  to  note  here  was  that 
the  researchers  developed  their  own  concepts  and  meanings  (such  as  alcoholism). 

Respondents  respond  to  a  researcher’s  question  or  complete  a  questionnaire.  TTiey  are 
respondents  because  they  have  answered  queries  built  in  the  researcher’s  semantics.  One  easy 


^  We  acknowledge  that  the  Intelligence  Community  and  others  have  a  different  traditional  use  of  the  word  informant. 
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example  of  this  is  the  census  survey,  which  has  historically  asked  respondents  to  categorize 
themselves  hy  race  categories  that  have  not  always  fit  the  self-identity  of  the  respondents.  Those 
census  questions  used  categories  of  the  researchers,  not  of  the  respondents. 

Actors  are  observed.  They  can  range  from  gorillas  to  Supreme  Court  justices  during  a  trial. 
In  each  of  these  examples,  the  researcher  is  either  unahle  or  unwilling  to  involve  himself  in  the 
process  he  is  analyzing.  Instead,  the  actors  are  observed  through  the  lens  of  the  researcher,  who 
uses  his  or  her  own  expertise,  terminology,  and  defined  categories  to  describe  the  actors.  For 
example,  a  researcher  with  medical  expertise  would  likely  make  different  observations  while 
observing  surgical  operating  room  procedures  than  would  a  researcher  lacking  that  expertise. 

Returning  to  informants,  a  researcher  using  an  informant  learns  from  the  informant  how 
things  are  defined  and  categorized.  This  is  what  anthropology  prides  itself  on.  This  is  not  to  say 
that  anthropologists  do  not  have  some  hypotheses  when  they  go  out  into  the  field,  but  they  are 
wary  of  how  much  they  are  dictating  the  content  and  process  of  the  research  and  how  much 
they  are  deriving  from  the  way  the  informant  views  his  or  her  own  culture. 

Thus,  a  project  can  have  all  or  some  of  these  data  sources.  For  example,  during  a  RAND 
study  of  women  in  the  military,  we  used  both  informants,  who  talked  to  us  of  their  perspec¬ 
tives  on  women  in  the  military  during  focus  groups,  and  respondents,  who  answered  surveys 
that  we  constructed  prior  to  analyzing  the  results  of  the  focus  groups.  Because  the  people  who 
participated  in  the  focus  groups  also  completed  a  survey,  these  same  individuals  were  both 
informants  and  respondents,  which  enriched  the  quality  and  the  depth  of  the  data  we  col¬ 
lected.  As  respondents,  they  answered  questions  in  our  words,  which  permitted  us  to  quantify 
the  answers  to  specific  issues  that  we  knew  in  advance  would  be  important.  As  informants, 
they  provided  rich  descriptions  of  their  perceptions  and  experiences,  which  were  a  valuable 
complement  to  the  quantitative  data. 
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Overview 

•  Introduction  to  data  collection  methods 

•  Applications  from  the  real  world 

•  Semi-structured  interviews 

•  Focus  groups 

•  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 
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We  begin  with  a  brief  overview  of  data  collection  methods. 


6  Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups 


Defining  Different  Types  of  Data  Coiiection 

•  Survey 

•  Interviews 

•  Focus  groups 

•  Observation 

•  (Data)  extraction 

•  Secondary  data  sources 
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There  are  many  forms  of  data  collection;  for  the  purposes  of  this  course,  we  define  these  modes 
as  follows. 

Surveys  are  fixed  sets  of  questions  that  can  be  administered  by  paper  and  pencil,  as  a  Web 
form,  or  by  an  interviewer  who  follows  a  strict  script. 

Interviews  are  discussions,  usually  one-on-one  between  an  interviewer  and  an  individ¬ 
ual,  meant  to  gather  information  on  a  specific  set  of  topics.  Interviews  can  be  conducted  in 
person  or  over  the  phone.  Interviews  differ  from  surveys  by  the  level  of  structure  placed  on  the 
interaction. 

Focus  groups  are  dynamic  group  discussions  used  to  collect  information. 

Observation  is  data  collection  in  which  the  researcher  does  not  participate  in  the  inter¬ 
actions.  Examples  of  this  include  observing  operating  room  procedures  or  Supreme  Court 
proceedings.  However,  it  should  be  noted  that  the  researcher’s  very  presence  may  have  some 
influence  on  the  participants  and  exchanges.  For  example,  while  the  researcher  is  unlikely  to 
influence  a  surgeon  or  a  Supreme  Court  justice,  it  is  not  difficult  to  imagine  the  researcher’s 
presence  influencing  other  participants,  such  as  small  children  at  play. 

Extraction  is  the  collection  of  data  from  documents,  records,  or  other  archival  sources. 
This  generally  includes  using  an  abstraction  process  to  cull  the  information  desired  from  the 
source.  Examples  of  this  might  be  collecting  information  on  dates  of  diagnoses  from  medical 
records  or  decision  dates  from  legal  records. 
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Secondary  data  sources  are  datasets  that  are  already  in  existence,  such  as  census  data. 
Researchers  may  select  variables  to  use  in  their  analysis  from  one  secondary  data  source  or  may 
combine  data  from  across  sources  to  create  new  datasets. 
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Strengths/Benefits  - 
How  to  Select  the  Method? 

*  Redundancy  with  other  studies 

*  Logistics 

-  Costs 

-  Time  available 

-  Staffing 

-  Access  to  records 

*  Sampling 

-  Frame  availability 

-  Sampling  size 

*  Institutional  Review  Board  (IRB)  and  Office  of 
Management  and  Budget  (0MB) 

-  Human  subjects  considerations 

RAND 


Many  considerations  go  into  deciding  the  appropriate  method  of  data  collection  to  use,  or 
even  if  data  collection  is  appropriate,  for  answering  the  research  questions. 

The  first  questions  that  you  should  ask  are:  Has  this  been  done  before?  Do  these  data 
already  exist?  If  so,  is  there  value-added  in  doing  this  again? 

There  also  are  many  logistical  considerations  that  need  to  be  addressed:^ 

•  What  will  the  data  collection  cost?  Can  the  project  afford  this  mode  of  data  collection? 

•  Is  there  enough  time  to  complete  the  data  collection?  There  are  many  steps  involved  in 
a  research  project — design,  approvals,  sampling,  conducting  the  fieldwork,  analysis,  and 
report  writing — and  each  step  can  take  considerable  time.  Obtaining  a  preexisting  data 
file  may  be  markedly  quicker  than  mounting  a  data  collection  effort. 

•  Staffing  must  be  considered.  Who  is  going  to  do  the  work,  and  are  they  trained  to  do 
it  properly?  How  many  people  are  needed  to  accomplish  this  data  collection  in  the  time 
available  to  complete  it? 

•  Are  the  people,  records,  or  information  needed  accessible?  For  instance,  think  about 
trying  to  conduct  a  survey  of  physicians.  Most  physicians  are  extremely  busy  and  con¬ 
sider  their  time  to  be  highly  valuable.  Collecting  data  from  physicians  takes  a  lot  of  effort 
and  is  not  always  successful. 


^  Adapted  from  Arksey  and  Knight  (1999). 
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If  the  data  collection  should  he  done,  and  you  can  resolve  the  logistical  considerations, 
then  you  need  to  consider  the  sampling  issues: 

•  Is  there  a  sampling  frame  that  exists  for  the  population  you  are  interested  in  studying?  If 
not,  do  you  have  the  ability  and/or  time  to  create  one?  How  do  you  decide  who  is  to  he 
included  in  your  population? 

•  How  large  of  a  sample  do  you  need?  Here  you  have  to  consider  what  you  want  to  he  able 
to  say  about  the  data.  How  generalizable  do  you  want  your  results  to  be? 

Finally,  there  are  issues  related  to  approvals.  Is  approval  needed  from  either  an  Institu¬ 
tional  Review  Board  (IRB)  or  the  federal  Office  of  Management  and  Budget  (OMB)?  IRB 
review  is  related  to  human  subjects  considerations  and  is  designed  to  identify  both  the  risk  to 
the  individual  participant  and  the  likelihood  of  that  risk  occurring.  OMB  review  is  designed 
to  ensure  that  the  data  collection  is  not  extant,  to  minimize  the  burden  of  survey  participation 
on  the  public,  and  to  minimize  the  burden  of  data  collection  cost  to  the  U.S.  government. 

Other  agencies  might  require  additional  approvals.  This  can  add  months  to  your  timeline. 
And  usually  you  need  to  have  your  materials  finalized  when  you  go  in  for  review. 
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Why  Focus  Groups  vs.  Interviews  vs. 

Surveys? 

•  Depth  of  information  (I,  FG) 

•  Resolve  seemingly  conflicting  information 
(FG,  I  with  specific  questions) 

•  Determine  relative  emphasis  on  issue 
(S,  I  with  specific  questions) 

•  Generalizability  (S,  I*) 

•  Timeliness  (not  S) 

•  Avoid  being  “another  survey”  (I,  FG) 

•  Sensitivity  of  issue  (I,  S) 

•  Classification  of  issue  (I,  S) 

•  Large  number  of  interviews 
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Often  when  selecting  the  method  for  primary  data  collection,  researchers  query  whether  focus 
groups,  interviews,  or  surveys  will  he  most  appropriate  to  the  research  effort.  There  are  many 
factors  to  consider  in  this  decision.  This  slide  presents  some  of  these  considerations  and  indi¬ 
cates  the  appropriateness  of  different  methods. 

In  seeking  a  very  complete  response,  interviews  and  focus  groups  are  most  likely  to  pro¬ 
vide  the  depth  of  information  that  might  he  useful.  Focus  groups  and  interviews  are  also 
the  best  methods  to  resolve  seemingly  conflicting  information,  because  the  researcher  has  the 
direct  opportunity  to  ask  about  the  apparent  conflict. 

When  interested  in  determining  the  relative  emphasis  on  an  issue,  that  is,  how  strongly 
someone  holds  an  opinion,  both  surveys  and  interviews  permit  the  researcher  to  ask  for  empha¬ 
sis.  The  researcher  might  directly  ask  someone  how  strongly  they  feel,  or  might  ask  them 
to  prioritize  issues  or  assign  weights  to  different  issues.  Focus  groups  are  less  appropriate  to 
determine  emphasis,  in  part  because  the  members  may  not  share  the  same  emphasis,  and  also 
because  the  group  dynamics  may  imply  an  emphasis  that  is  misleading. 

Data  are  more  likely  to  be  generalizable  if  they  are  well  sampled  and  collected  by  surveys 
or  by  a  large  number  of  interviews.  Focus  group  data  are  never  generalizable  beyond  the  groups 
conducted. 

When  the  research  effort  is  time-constrained,  surveys  may  be  least  appropriate  because 
survey  efforts  often  take  longer  to  field  correctly. 

One  benefit  of  conducting  interviews  or  focus  groups,  especially  with  over-sampled  popu¬ 
lations,  is  that  individuals  in  those  populations  have  sometimes  grown  weary  of  surveys  and 
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appreciate  the  opportunity  to  express  their  opinions  and  experiences  in  person,  rather  than  in 
another  pen-and-paper  survey. 

The  sensitivity  of  an  issue  could  also  discourage  the  use  of  focus  groups,  as  the  group  con¬ 
text  may  not  he  appropriate  to  discuss  sensitive  issues.  Likewise,  if  the  issue  is  classihed,  focus 
groups  will  also  likely  he  inappropriate,  as  the  participants  would  need  to  share  not  only  the 
correct  access,  hut  also  the  need  to  know  the  material. 
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Complementary  Methods 

•  Methods  used  in  sequence 

-  Provide  explanation 

-  Used  in  instrument  development 

*  Methods  used  in  concert 

-  Complementary  data  collection 

-  Access  issues,  especially  with  senior  people 


RAND 


In  actual  projects,  it  is  often  the  case  that  multiple  methods  are  used  together. 

Certain  methods  may  he  used  in  sequence — for  instance,  qualitative  analysis  provided 
hy  focus  groups,  interviews,  or  observations  might  add  to  the  interpretation  of  results  found 
hy  analyzing  previously  collected  survey  data.  Also,  focus  groups  and  interviewing  are  often 
employed  as  a  part  of  a  questionnaire-design  process. 

Methods  can  also  he  used  in  concert.  This  is  particularly  the  case  when  there  are  con¬ 
cerns  about  targeting  a  data  collection  method  to  certain  types  of  individuals,  or  when  you  are 
trying  to  obtain  different  kinds  of  information  from  various  sources.  For  instance,  high-status 
individuals  may  be  unlikely  to  respond  to  a  survey,  so  the  researcher  may  be  more  successful 
if  he  or  she  attempts  to  conduct  interviews  with  them,  although  it  may  be  very  difhcult  to 
obtain  time  on  a  senior  person’s  calendar.^  Other  projects  might  include  interviews  with  senior¬ 
ranking  individuals  from  an  organization  and,  simultaneously,  surveys  of  the  more  junior  per¬ 
sonnel.  Using  multiple  methods  can  enhance  the  research  project,  but  the  important  thing  to 
remember  is  that,  regardless  of  the  methods  chosen,  quality  research  includes  collecting  quality 
data. 


^  Difficulties  in  obtaining  interview  time  with  senior  personnel  or  convincing  them  to  complete  a  survey  can  be  addressed 
with  encouraging  messages  from  endorsers  even  more  senior  to  the  respondent,  if  the  research  has  the  luxury  of  such  senior 
support. 
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Overview 

•  Introduction  to  data  collection  methods 

•  Applications  from  the  real  world 

•  Semi-structured  interviews 

•  Focus  groups 

•  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 


RAND 


The  next  section  of  this  course  provides  an  example  of  two  projects  that  have  used  a  variety  of 
data  collection  techniques. 
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RAND 


NATIONAL  DEFENSE  RESEARCH  INSTITUTE 


New  Opportunities  for  Military  Women  (1997) 

and 

Assessing  the  Assignment  Policy  for 
Army  Women  (2007) 


It  is  worthwhile  to  take  a  brief  sojourn  from  methodology  to  provide  examples  of  two  projects 
that  used  mixed  methods  for  data  collection.  These  example  research  projects  studied  similar 
topics:  the  effect  of  gender  integration  on  the  military  and  the  extent  to  which  Army  opera¬ 
tions  in  Iraq  were  consistent  with  the  assignment  policy  for  Army  women.  They  are  provided 
as  examples  of  projects  that  use  multiple  data  collection  methods.'' 


The  two  studies  are  Margaret  C.  Harrell  and  Laura  Miller,  New  Opportunities  for  Military  Women:  Effects  Upon  Readi¬ 
ness,  Cohesion,  and  Morale,  Santa  Monica,  Calif.:  RAND  Corporation,  MR-896-OSD,  1997;  and  Margaret  C.  Harrell, 
Laura  Werber  Castaneda,  Peter  Schirmer,  Bryan  W.  Hallmark,  Jennifer  Kavanagh,  Daniel  Gershwin,  and  Paul  Steinberg, 
Assessing  the  Assignment  Policy  for  Army  Women,  Santa  Monica,  Calif:  RAND  Corporation,  MG-590-1-OSD,  2007.  Both 
studies  are  available  free  for  download  on  the  RAND  Web  site. 
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Research  Goals  and  Question  Set: 
New  Opportunities  for  Military  Women 

•  Evaluate  the  progress  of  integration  in  newly  opened 
occupations  and  units 

*  Determine  the  effect  of  gender  integration 


*  What  occupations  include  women? 

*  To  which  units  are  women  assigned? 

*  What  gender  issues  have  arisen  in  those  units? 

*  How  has  leadership  dealt  with  those  issues? 

*  How  important  are  those  issues  for  readiness,  cohesion, 
morale? 


RAND 


The  first  of  our  two  example  projects,  Harrell  and  Miller  (1997),  was  a  congressionally  man¬ 
dated  study.  The  research  was  conducted  after  changes  in  the  law  opened  new  occupations 
to  military  women  and  also  permitted  women  to  serve  on  combatant  ships  and  in  combat 
aircraft.  This  project  addressed  two  main  issues:  (1)  the  progress  of  integration  in  the  newly 
opened  occupations  and  units  and  (2)  the  effect  of  gender  integration  upon  those  units.  This 
slide  provides  the  research  questions  indicated  by  Congress  to  address  these  issues. 
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Mixed  Approach:  Each  Method  Addressed  a 
Different  Aspect  of  the  issue 

•  Data  analysis  of  military  personnel  files  quantified 
the  progress  of  gender  integration  in  occupations 
and  in  units 

•  Interviews  obtained  the  perspectives  of  unit 
leadership  in  14  units 

*  Focus  groups  (492  participants)  permitted  rich 
discussions  of  different  gender  issues  in 
integrated  units 

*  934  surveys  provided  an  assessment  of  the  extent 
to  which  gender  issues  were  a  problem 


RAND 


Each  data  collection  method  addressed  a  different  aspect  of  the  research.  The  project  team 
analyzed  the  data  from  military  personnel  hies  in  order  to  quantify  the  number  and  propor¬ 
tion  of  women  in  newly  opened  occupations  and  in  newly  opened  units.  Interviews  with  unit 
leadership  from  the  14  selected  units  permitted  discussion  about  gender  issues  in  units  and  the 
approach  of  leadership  to  deal  with  such  issues.  The  questions  asked  in  these  interviews  were 
very  similar  to  the  questions  posed  in  the  focus  groups.  However,  the  unit  leadership  could  not 
have  been  included  in  the  focus  groups  at  the  unit  locations,  since  their  participation  would 
have  skewed  and  reduced  the  free  interaction  of  the  focus  group  discussions. 

Focus  groups  were  conducted  to  discuss  the  different  gender  issues  that  had  occurred  in 
the  newly  integrated  units.  The  focus  groups  were  divided  at  each  unit  location  by  pay  grade 
group  and  also  by  gender.  Finally,  surveys  were  also  administered  to  all  focus  group  partici¬ 
pants,  as  well  as  to  all  additional  unit  personnel  that  were  available  to  complete  a  survey.  The 
surveys  asked  broader  questions  about  issues  affecting  readiness,  cohesion,  and  morale. 
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Each  Method  Provided  a 
Different  Aspect  of  the  Answer 

•  Data  analysis 

-  Unit  integration  proceeds  quickly 

-  Occupational  integration  is  siower  process 

•  Interviews 

-  Not  a  readiness  issue 

-  Most  gender  issues  can  be  addressed  by  leadership 

•  Focus  groups 

-  Both  positive  and  negative  effects  of  gender  integration 

•  Surveys 

-  Leadership  and  training  have  considerabiy  more  effect  (than 
gender)  on  readiness,  cohesion,  and  moraie 


RAND 


Each  research  method  provided  a  different  aspect  of  the  final  research  findings. 

The  data  analysis  permitted  realization  that  unit  integration  is  a  result  of  assignment. 
Since  women  can  begin  to  he  assigned  to  a  unit  as  soon  as  the  decision  is  made  to  integrate 
a  unit,  unit  integration  can  proceed  relatively  quickly.  Integrating  occupations,  on  the  other 
hand,  requires  that  women  complete  the  respective  training  pipelines.  Since  it  can  take  a  long 
time  to  train  a  new  fighter  pilot,  occupational  integration  proceeds  at  a  slower  rate  than  does 
unit  integration. 

The  interviews  suggested  that  gender  integration  was  not  a  readiness  issue.  Addition¬ 
ally,  the  commanders  interviewed  felt  equipped  and  informed  to  handle  any  gender  issues 
that  arose  in  their  units.  Further,  these  interviews  provided  researchers  with  examples  of  how 
leaders  were  handling  perceived  gender  issues.  For  example,  when  the  work  was  physically 
demanding,  such  as  unloading  supplies,  the  commanders  did  not  urge  women  to  lift  as  much 
as  the  men  did,  hut  the  leaders  did  ensure  that  women  carried  additional  loads  to  compensate 
for  their  lighter  loads. 

The  focus  groups  conducted  for  this  research  effort  provided  very  detailed  explanations  of 
gender  issues  in  integrated  units  and  also  rich  examples  of  both  the  positive  and  the  negative 
effects  of  gender  integration. 

The  surveys  placed  the  discussion  about  gender  effects  into  the  broader  context  of  these 
units.  Specifically,  the  surveys  included  open-ended  questions  asking  respondents  to  list  the 
factors  that  affect  readiness,  that  affect  cohesion,  and  that  affect  morale.  Although  the  respon- 


18  Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups 


dents  were  all  aware  that  the  study  was  focused  on  gender,  very  few  of  the  survey  answers 
attributed  readiness,  cohesion,  or  morale  to  gender  issues. 

This  project  succeeded  because  of  the  complementary  mix  of  data  collection  methods.  The 
data  analysis  provided  the  necessary  quantitative  results.  The  interviews  provided  the  research¬ 
ers  with  knowledge  from  the  level  of  the  unit  commanders.  The  focus  groups  provided  very 
rich  description  and  examples,  and  the  surveys  permitted  the  researchers  to  place  those  colorful 
discussions  in  a  broader  context.  Thus,  to  the  extent  that  gender  integration  was  an  issue  for 
units,  the  focus  groups  provided  insights  about  the  types  of  issues  confronted.  However,  the 
survey  findings  suggested  that  gender  integration  was  not  having  a  significant  effect  on  readi¬ 
ness,  cohesion,  or  morale. 
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Research  Goals  and  Question  Set: 
Assessing  the  Assignment  Policy 

•  Determine  whether  the  Army  was  complying  with 
the  assignment  policy  for  military  women 


*  What  did  the  assignment  policy  mean;  was  there  a 
shared  interpretation? 

*  How  many  women  have  been  deployed  to  Iraq? 

*  To  which  units  are  Army  women  assigned? 

*  What  are  those  units,  and  Army  women,  doing  in 
Iraq? 


RAND 


The  second  of  our  example  studies  (Harrell  et  ah,  2007)  was  also  mandated  by  Congress  and 
was  established  to  determine  whether  the  Army  was  complying  with  the  assignment  policies  for 
Army  women,  given  the  operations  in  Iraq.  The  research  questions  are  listed  above.  This  study 
took  an  anthropological  approach — it  began  by  deconstructing  the  assignment  policy  and 
questioning  whether  everyone  had  the  same  interpretation  of  the  policy.  The  study  also  quanti¬ 
fied  the  deployments  of  women  to  Iraq,  and  asked  what  units  Army  women  were  assigned  to  as 
well  as  what  those  units,  and  thus  Army  women,  were  actually  doing  in  Iraq. 
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Mixed  Approach:  Each  Method  Addressed  a 
Different  Aspect  of  the  issue 

•  Data  analysis  of  four  “snapshots”  of  deployed  personnel  in  Iraq 
permitted  anaiysis  of  numbers  and  trends  of  Army  women 
deployed  to  Iraq 

-  Also  requested  and  analyzed  numbers  and  occupations  of  women 
who  received  the  Combat  Action  Badge 

*  16  interviews  with  OSD,  Joint  Staff,  and  Army  leadership  and 
members  of  Congress  to  establish  the  intent/objectives  and  the 
meaning  of  the  assignment  policy 

*  8  interviews  with  unit  leadership  to  understand  where  women 
were  assigned  and  what  those  units  did  in  Iraq 

•  16  focus  groups  (80  participants)  to  discuss  where  women 
were  assigned  and  what  those  units  did  in  Iraq 

RAND 


This  project  included  the  data  analysis  of  four  “snapshots”  of  deployed  personnel  in  Iraq, 
which  permitted  the  researchers  to  quantify  the  numbers  and  proportions  of  female  personnel 
deployed  to  Iraq.  The  research  team  also  requested  and  analyzed  data  regarding  the  women 
who  had  been  awarded  the  Combat  Action  Badge. 

The  research  team  conducted  11  interviews  with  Army,  Office  of  the  Secretary  of  Defense 
(OSD),  and  Joint  Staff  leadership,  as  well  as  5  additional  interviews  with  some  members  of 
Congress.  The  purpose  of  these  interviews  was  to  discuss  the  intent  or  objectives  of  the  assign¬ 
ment  policy  and  to  interpret  the  meaning  of  the  policy. 

The  project  team  also  traveled  to  selected  units  that  had  recently  returned  from  Iraq. 
While  visiting  the  units,  the  project  team  conducted  interviews  with  the  unit  leadership  and 
focus  groups  with  unit  personnel.  The  questions  asked  in  the  interviews  and  the  focus  groups 
were  similar,  but  the  unit  commanders  were  interviewed  individually.  The  interviews  and  focus 
groups  addressed  the  units  to  which  women  had  been  assigned  and  the  missions  performed  by 
those  units  while  in  Iraq. 


Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups  21 


Each  Method  Provided  a 
Different  Aspect  of  the  Answer 

*  Data  analysis 

-  Women  comprised  10%  of  enlisted  and  12%  of  officers 
deployed 

-  Steady  level  of  representation;  no  trends  over  time 

-  1,777  received  CAB;  included  HR,  MP,  supply,  truck  drivers, 
cooks 

*  Interviews  with  OSD,  JS,  Army  leadership,  Congress 

-  No  shared  interpretation  of  the  assignment  policy 

*  Interviews  with  unit  leadership  and  focus  groups 

-  Women  are  assigned  to  appropriate  units  for  DoD  policy 

-  Army  may  be  violating  Army  policy 

-  Women  are  in  combat 

-  No  understanding  in  theater  of  meaning  of  assignment 
policy 

RAND 


The  data  analysis  indicated  that  women  constituted  10  percent  of  the  enlisted  personnel  and  12 
percent  of  the  officers  deployed  to  Iraq,  and  that  women  were  steadily  represented;  there  were 
no  directional  trends  over  time.  Additionally,  1,777  women  had  received  the  Combat  Action 
Badge,  including  women  in  a  variety  of  occupations,  including  human  resources,  military 
police,  supply,  truck  drivers,  and  cooks. 

The  interviews  with  OSD,  Army,  and  Joint  Staff  leadership  and  with  members  of  Con¬ 
gress  indicated  that  there  was  no  shared  interpretation  of  the  assignment  policy  among  these 
leaders.  While  many  respondents  were  confident  that  they  understood  the  policy,  the  indi¬ 
vidual  interpretations  varied. 

The  interviews  with  unit  leadership  and  the  focus  groups  with  unit  personnel  indicated 
that  Army  women  had  been  assigned  to  appropriate  units  consistent  with  the  Department  of 
Defense  (DoD)  policy.  However,  the  Army  policy  differs  from  the  DoD  policy,  and  includes 
additional  specifications  regarding  the  units  to  which  women  can  be  assigned.  Thus,  it  became 
evident  from  our  interviews  and  focus  groups  that,  although  the  Army  was  adhering  to  the 
DoD  assignment  policy,  it  was  possible  that  the  Army  was  in  violation  of  its  own  assignment 
policy.  The  sessions  confirmed  that  women  were  clearly  in  combat  situations,  although  that  is 
not  itself  a  violation  of  the  assignment  policy.  Further,  the  interviews  and  focus  groups  indi¬ 
cated  that,  like  the  military  leadership,  more  junior  deployed  personnel  also  did  not  under¬ 
stand  the  meaning  of  the  assignment  policy. 
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In  closing,  this  second  example  project  also  indicates  how  different  research  methods 
addressed  different  aspects  of  the  research  questions.  These  two  projects  are  examples  of  real- 
world  applications  of  qualitative  research  methods. 
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Overview 

•  Introduction  to  data  collection  methods 
'Applications  from  the  real  world 
•Semi-structured  interviews 

•  Focus  groups 

•  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 


RAND 


We  now  proceed  to  discuss  these  methods  in  more  detail,  beginning  with  semi-structured 
interviews. 
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Interviews  Can  Obtain  Different  Kinds  of  Data 

*  Interviews  to  gather  opinions,  perceptions, 
attitudes 

*  Interviews  to  gather  background  information 

-  Expert  knowledge 

-  Facts,  descriptions  of  processes 

*  Some  interviews  may  include  aspects  of  both 


RAND 


Researchers  use  interviews  for  a  variety  of  purposes.  Interviews  can  be  used  as  a  primary  data 
gathering  method  to  collect  information  from  individuals  about  their  own  practices,  beliefs,  or 
opinions.  They  can  be  used  to  gather  information  on  past  or  present  behaviors  or  experiences. 

Interviews  can  further  be  used  to  gather  background  information  or  to  tap  into  the  expert 
knowledge  of  an  individual.  Think  here  about  interviewing  a  subject-matter  expert  on  a  new 
policy.  These  interviews  will  likely  gather  factual  material  and  data,  such  as  descriptions  of 
processes. 

Interviews  will  often  include  the  collection  of  both  types  of  information. 

The  difference  between  these  types  of  interviews  is  readily  apparent  to  most.  However, 
some  researchers  err  on  the  side  of  not  considering  the  gathering  of  background  information  or 
expert  knowledge  as  an  interview,  and  therefore  do  not  pay  close  attention  to  the  way  questions 
are  asked  in  such  interviews.  This  can  compromise  the  quality  of  the  data  collected. 
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Amount  of  Control  Exercised  Determines 
Place  on  Interview  Continuum 


Control 


Kind  of  Interview 


RAND 


Interviews  can  be  placed  on  a  continuum  of  structure,  from  “unstructured”  to  highly  “struc¬ 
tured.”  Imbedded  in  this  continuum  is  the  idea  of  how  much  “control”  the  interviewer  will 
have  over  the  interaction.  There  are  benefits  to  each  of  these  kinds  of  interviews. 
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Amount  of  Control  Exercised  Determines 
Place  on  Interview  Continuum 

Control 


Kind  of  Interview 


■  I’m  here  for  the  year  to  understand  how  you  train  your  personnel. 

RAND 


With  unstructured  interviews,  the  researcher  has  a  clear  plan,  hut  minimum  control  over  how 
the  respondent  answers.  An  example  might  he  a  case  where  a  researcher  visits  an  office,  sits 
down  with  someone  who  works  there,  and  asks,  “What  do  you  do?” 

The  conversation  can  go  in  many  directions,  and  will  vary  much  hy  the  respondent.  The 
interviewer  does  not  exert  much  control  over  the  course  of  the  discussion.  He  or  she  might 
follow  explanations  with  additional  questions  based  on  the  topics  that  the  respondent  brings 
up,  but  the  session  would  be  relatively  free-flowing. 

Not  surprisingly,  gathering  information  in  this  manner,  though  it  might  lead  to  very  rich 
and  nuanced  data,  can  take  a  long  time.  These  types  of  interviews  are  really  most  suitable  when 
researchers  have  a  great  deal  of  time  to  spend  with  the  community  they  are  studying. 

The  statement  provided  on  this  slide,  “I’m  here  for  the  year  to  understand  how  you  train 
your  personnel,”  is  an  example  of  how  one  might  open  an  unstructured  interview.  We  will  con¬ 
trast  this  example  with  semi-structured  and  structured  examples  in  subsequent  slides. 
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Amount  of  Control  Exercised  Determines 
Place  on  Interview  Continuum 

Control 


Kind  of  Interview 


•  Could  you  describe  the  various  kinds  of  training  you  use? 

•  PROBE:  Who  develops  your  training? 

•  PROBE:  How  often  do  you  adjust  your  training? 

•  PROBE:  Who  receives  your  training? 

RAND 


Semi-structured  interviews  are  used  often  in  policy  research.  In  semi-structured  interviewing, 
a  guide  is  used,  with  questions  and  topics  that  must  he  covered.  The  interviewer  has  some  dis¬ 
cretion  about  the  order  in  which  questions  are  asked,  hut  the  questions  are  standardized,  and 
prohes  may  he  provided  to  ensure  that  the  researcher  covers  the  correct  material.  This  kind  of 
interview  collects  detailed  information  in  a  style  that  is  somewhat  conversational.  Semi-struc¬ 
tured  interviews  are  often  used  when  the  researcher  wants  to  delve  deeply  into  a  topic  and  to 
understand  thoroughly  the  answers  provided. 

The  example  question  on  the  slide  indicates  an  initial  question  that  would  he  asked  about 
training,  as  well  as  probes  that  the  researcher  might  use  to  ensure  that  complete  and  consistent 
information  was  received  across  different  interviews. 
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Amount  of  Control  Exercised  Determines 
Place  on  Interview  Continuum 


Control 


Kind  of  Interview 


■  What  training  materiais  does  your  organization  use? 

O  Materials  prepared  by  the  agency 
O  Materials  adapted  from  military  services 
O  Materials  created  in  this  organization 
O  Other 

RAND 


The  most  controlled  type  of  interview  is  structured.  In  structured  interviews,  the  questions 
are  fixed  and  they  are  asked  in  a  specific  order.  Multiple  respondents  will  be  asked  identical 
questions,  in  the  same  order.  Structured  interviews  most  closely  approximate  a  survey  being 
read  aloud,  without  deviation  from  the  script.  Structured  interviews  have  several  advantages 
over  surveys  including  lower  levels  of  item  nonresponse  and  the  ability  for  an  interviewer  to 
mitigate  inappropriate  responses  (see  Fowler,  2002,  for  a  more  thorough  discussion).  However, 
in  a  structured  interview,  if  a  respondent  indicates  that  they  do  not  understand  a  question 
or  a  term  in  the  question,  the  interviewer  is  generally  limited  to  providing  only  a  previously 
scripted  explanation  or  defining  the  term  as  “Whatever  [the  term]  means  to  you.”  Otherwise, 
the  interviewer  is  generally  unable  to  provide  any  explanation  beyond  repeating  the  question. 
These  interviews  are  often  used  when  one  has  very  large  samples  and  is  looking  for  data  that 
can  be  generalized  to  a  large  population. 

In  the  example  provided,  the  interviewer  would  read  the  training  question  aloud  and  then 
read  the  response  choices  to  the  respondent. 
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Overview 

•  Introduction  to  data  collection  methods 
"Applications  from  the  real  world 

"Semi-structured  interviews 
—  Frame  the  research 
—  Sampling 

-  Designing  questions  and  probes 

-  Deveioping  the  protocoi 

—  Preparing  for  the  interview 
—  Conducting  the  interview 

-  Capturing  the  data 

" Focus  groups 

"  Brief  introduction  to  data  management  and  analysis 
"  Closing  remarks 

RAND 


In  the  next  section,  we  discuss  framing  the  research  and  sampling. 
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Frame  the  Research 

*  Formalize  your  research  question(s) 

-  What  do  you  hope  to  learn? 

-  Who  needs  these  insights? 

*  Identify  best  source(s)  of  information  for  each 
research  question 

-  May  be  more  than  one 

*  Determine  number  of  people  you  need  to  speak 
with 

-  Subgroups 


RAND 


Before  a  researcher  can  decide  what  types  of  informants  or  respondents  to  include  in  a  research 
project  and  what  to  ask  them,  the  main  research  questions  need  to  he  identified/  In  other 
words,  what  does  the  researcher  hope  to  learn?  The  researcher  should  take  into  account  the 
audience  and  what  he  or  she  wants  to  he  able  to  tell  them. 

Then,  the  researcher  should  consider  all  the  possible  sources  of  knowledge  or  information 
appropriate  to  answer  the  specihc  questions.  It  might  be  appropriate  to  use  multiple  methods 
to  gather  the  data.  For  example,  the  military  women  projects  discussed  earlier  used  both  focus 
groups  and  interviews. 

Finally,  the  researcher  needs  to  consider  the  different  participant  types.  In  the  project 
discussed  earlier,  military  personnel,  policymakers,  and  lawmakers  were  all  included.  The 
researcher  should  think  about  the  characteristics  that  differentiate  the  participants  and  the 
resultant  subgroups.  For  example,  the  researcher  may  decide  to  include  both  junior  and  senior 
military  personnel.  In  the  case  of  the  military  women  projects,  the  military  personnel  were 
subdivided  into  different  subgroups;  data  were  collected  from  military  commanders  with  inter¬ 
views  while  data  were  collected  from  other  military  personnel  with  focus  groups.  We  will  dis¬ 
cuss  this  in  more  detail  later  in  this  course. 


7 


Adapted  from  Stewart  and  Shamdasani,  1990. 
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Basic  Primer  of  Sampiing 

•  Random 

•  Systematic;  stratified 

•  Structured 

•  Cluster 

•  Judgment 

•  Convenience 

•  Opportunity 

•  Snowball 

Get  sampling  help! 

RAND 


An  important  part  of  any  research  effort  is  determining  the  sampling.  This  material  is  not 
intended  to  provide  complete  expertise  in  sampling,  hut  rather  to  define  and  explain  the  dif¬ 
ferent  types  of  sampling. 

Ideally,  if  appropriate  to  the  research  question,  a  research  effort  would  randomly  sample. 
However,  keep  in  mind  that  a  so-called  random  sample  rarely  is  truly  random.  First,  research 
generally  randomly  samples  from  a  defined  population,  such  as  particular  area  codes  or  par¬ 
ticular  military  units,  schools,  etc.  The  defined  nature  of  the  population  itself  precludes  total 
randomness.  Second,  the  contact  method  also  removes  some  randomness.  For  example,  if 
respondents  are  to  he  contacted  hy  telephone  landlines,  the  portion  of  the  population  that  does 
not  have  a  landline  is  inherently  excluded.  A  researcher  may  not  he  able  to  address  these  issues, 
hut  he  or  she  should  acknowledge  such  weaknesses  in  the  study  sample’s  “randomness.” 

Systematic  or  stratified  sampling  involves  first  stratifying  the  sample  based  on  its  char¬ 
acteristics  and  then  using  a  mathematical  process  to  determine  the  sampling,  generally  every 
nth  name  based  on  the  total  number  desired  in  the  sample.  This  method  is  often  used  in  a 
stratified  population,  such  as  the  military.  In  the  case  of  a  project  sampling  the  military,  the 
researcher  would  determine  how  many  participants  he  or  she  would  like  from  each  pay  grade, 
or  from  groupings  of  pay  grade,  such  as  junior  enlisted  personnel,  midgrade  enlisted  personnel, 
and  senior  enlisted  personnel.  The  researcher  would  divide  that  group  by  the  sample  number 
desired,  and  include  every  nth  name.  For  example,  if  the  total  group  included  5,000  names 


32  Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups 


and  the  researcher  wished  to  include  100  names  from  that  group,  the  researcher  would  include 
every  hftieth  name.  When  sampling  in  this  manner,  it  is  important  to  consider  how  the  names 
are  ordered.  For  example,  alphabetizing  names  will  group  similar  family  names  and,  poten¬ 
tially,  similar  ethnic  groups  that  might  share  similar  names,  together.  Likewise,  ordering  hy 
home  telephone  number  would  group  similar  area  codes  (and  thus  residential  areas)  together. 
One  solution  might  be,  for  example,  to  order  the  names  based  on  first  name  or  based  on  the 
seventh  digit  of  a  10-digit  telephone  number. 

Sampling  is  structured  in  some  research  so  that  the  research  either  does  not  overrepresent 
certain  groups,  or  in  other  cases,  to  purposively  oversample  certain  minority  groups.  For  exam¬ 
ple,  RAND  conducted  a  study  to  assess  why  there  was  not  greater  representation  of  minori¬ 
ties  among  elite  military  occupations  (Harrell  et  ah,  1999).  The  structured  sampling  approach 
was  employed  so  as  to  include  as  many  minorities  from  those  occupations  as  possible,  since 
their  perspective  was  important  to  the  research.  Another  sampling  method  would  likely  have 
included  relatively  few  minorities. 

Cluster  sampling  helps  reach  a  certain  population,  and  also  focuses  the  research  in  one 
geographic  area.  For  instance,  a  school,  retirement  community,  or  daycare  facility  for  dementia 
patients  can  be  selected  and  then  a  sample  from  these  populations  can  be  drawn,  which  might 
be  difficult  to  do  from  a  larger  population. 

Judgment  sampling  reflects  some  knowledge  of  the  topic,  so  that  people  whose  opinion 
will  be  important  to  the  research,  because  of  what  you  already  know  about  them,  will  be 
selected.  The  RAND  study  that  considered  the  assignment  of  Army  women  in  Iraq  (Harrell 
et  ah,  2007)  used  judgment  sampling  to  interview  a  female  battalion  commander  who  had 
recently  returned  from  Iraq.  Her  perspective  was  judged  valuable  to  the  research,  but  her  entire 
unit  had  not  been  selected  for  inclusion  in  the  study. 

Convenience  sampling  is  generally  used  in  the  testing  phase  of  research.  If,  for  example,  a 
researcher  plans  to  survey  young  mothers,  the  researcher  might  choose  to  test  the  survey  with 
some  young  mothers  in  her  neighborhood.  RAND  welcomes  a  selection  of  military  fellows 
each  year  to  study  at  RAND,  and  some  RAND  researchers  use  the  military  fellows  to  test 
interviews  or  surveys  focused  on  the  military. 

Opportunity  sampling  is  just  what  its  name  implies,  making  use  of  opportunities  as  they 
arise  and  interviewing  or  surveying  individuals  that  you  had  not  planned  to  encounter. 

Snowball  sampling  occurs  when  the  research  benefits  from  one  participant  suggesting  or 
introducing  another  participant  to  the  researcher.  While  most  research  projects  benefit  from 
some  amount  of  snowballing,  this  is  a  dangerous  primary  sampling  design,  as  the  risk  inherent 
in  snowball  sampling  is  the  overrepresentation  of  a  single,  networked  group. 

The  intended  emphasis  of  this  slide  is  to  define  and  explain  different  sampling  methods. 
However,  this  explanation  does  not  obviate  the  need  to  consult  statisticians  or  other  sampling 
experts  during  a  research  effort,  to  ascertain  the  strength  of  the  data  obtained  and  thus  the 
emphasis  that  can  be  placed  on  findings. 
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The  Benefits  of  Different  Sampiing  and  the 
Abiiity  to  Generaiize 


t 


Generalizability 

Claims 


Random 

Structured 


Inferences 

i 

RAND 


Snowball 

Opportunity 

Convenience 


This  slide  illustrates  some  of  the  relative  strengths  of  different  sampling  methods.^  At  the 
extreme,  if  researchers  only  consult  participants  obtained  by  a  convenience  method,  the  find¬ 
ings  will  be,  at  the  most,  only  inferences;  for  example,  if  five  young  mothers  provided  identi¬ 
cal  responses,  the  researcher  might  infer  about  the  broader  population  of  young  mothers,  but 
would  be  unable  to  make  any  stronger  assertions.  Opportunity  and  snowball  methods  provide 
only  slightly  stronger  results  than  does  a  convenience  sample.  At  the  other  end  of  the  spectrum, 
a  truly  random  sample  will  provide  the  most  generalizable  findings,  dependent  upon  the  size 
of  the  sample.  Likewise,  a  sufficiently  sized  and  well-conducted  structured  sample  will  also 
result  in  more  generalizable  claims  than  would  a  sample  (of  the  same  size)  obtained  by  other 
methods. 


Adapted  from  Arksey  and  Knight,  1999. 
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Overview 

•  Introduction  to  data  collection  methods 
'Applications  from  the  real  world 
'Semi-structured  interviews 
—  Frame  the  research 
—  Sampling 

-  Designing  questions  and  probes 

-  Deveioping  the  protocoi 

—  Preparing  for  the  interview 
—  Conducting  the  interview 

-  Capturing  the  data 

' Focus  groups 

'  Brief  introduction  to  data  management  and  analysis 
'  Closing  remarks 
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Our  discussion  will  now  turn  to  designing  questions  and  probes  for  a  semi-structured 
interview. 
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Question  Design  is  important 

*  Descriptive  questions 

*  Structural  questions 

*  Contrast  questions 
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There  are  different  types  of  questions,  with  different  goals.  In  the  most  general  sense,  descriptive 
questions  ask  people  to  describe  things  and  may  provide  insights  or  suggest  areas  for  query  that 
the  researcher  might  not  have  considered.  Structural  questions  help  the  researcher  understand 
relationships  between  things,  and  to  categorize  groups  of  like  things  or  like  processes.  Contrast 
questions  help  the  researcher  understand  what  terms  mean.  The  subsequent  slides  cover  these 
types  of  questions  in  more  detail.^ 


^  For  more  discussion  of  question  types  as  discussed  on  the  subsequent  slides,  see  Spradley,  1979. 
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Descriptive  Questions  Resuit  in  a  Narrative 

•  Grand  tour  questions  -  repeated  phrases 

•  Mini  tour  questions  -  more  focused 

•  Example  questions 

-  Please  give  me  an  example  of  something  he  does  to  make  you 

feel  that  way? 

•  Experience  questions 

-  Please  tell  me  something  you  did  while  you  were  deployed? 

•  Native  language  questions 

-  How  would  you  refer  to  new  trainees? 

-  What  do  you  call  people  who  do  that  kind  of  work? 

RAND 


It  is  important  for  the  researcher  to  determine  the  type  of  answer  that  he  or  she  would  like. 
For  example,  descriptive  questions  result  in  a  narrative,  and  they  should  only  he  asked  if  the 
researcher  wants  the  respondent  to  take  the  time  to  answer  with  a  narrative,  which  might  he 
lengthy.  Five  types  of  descriptive  questions  are  explained  here. 

The  grand  tour  question  is  a  good  type  of  question  to  use  near  the  beginning  of  an  inter¬ 
view,  because  it  often  encourages  a  respondent  to  speak.  A  grand  tour  question  might  be  rela¬ 
tively  simple,  and  sometimes  includes  multiple  small  questions  or  repeated  phrases.  An  exam¬ 
ple  might  be,  “I  am  interested  in  your  life  when  you  were  growing  up.  What  was  your  family 
like?  Where  did  you  live?  Where  and  for  how  long  did  you  go  to  school?”  This  is  a  grand  tour 
question,  because  it  involves  multiple  questions  and  leads  to  one  large  explanation.  Another 
example  might  be  asking  a  young  soldier  about  how  he  came  to  be  in  the  military:  “I’m  inter¬ 
ested  in  how  you  decided  to  come  into  the  military,  so  I’d  like  to  understand  how  your  life  was 
when  you  were  growing  up.  What  was  your  family  like?  Where  are  you  from?  How  did  you 
learn  about  the  military?  Why  did  it  sound  appealing  to  you?  How  did  you  finally  decide  to 
join  the  military?” 

The  mini  tour  question  asks  about  a  specific  element  and  is  often  used  to  follow  up  after  a 
grand  tour  question.  For  example,  “You’ve  told  me  a  lot  about  your  life  when  you  were  growing 
up,  but  you  haven’t  said  much  about  your  parents.  Please  tell  me  more  about  your  parents.  For 
example,  what  did  they  do  for  a  living?” 

Example  questions  ask  for  a  particular  example.  For  instance,  if  the  respondent  were  to 
comment  that  his  parents  were  very  strict  when  he  was  growing  up,  the  researcher  might  ask. 
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“Can  you  give  me  an  example  of  something  they  would  or  wouldn’t  do  that  made  you  feel  they 
were  strict  parents?” 

Experience  questions  ask  such  things  as  “Can  you  give  me  an  example  of  something  you 
did  while  you  were  deployed?”  Bear  in  mind  that,  to  the  extent  that  certain  experiences  come 
readily  to  mind,  they  may  not  he  representative.  If  a  soldier  has  an  exciting  story  about  the 
time  he  caught  a  foreign  national  scaling  a  fence  to  break  into  their  encampment  and  steal 
ammunition,  it  may  be  a  true  story,  but  also  may  not  be  representative  of  his  broader  deploy¬ 
ment  experience. 

Native  language  questions  ask  someone  to  use  his  or  her  own  terminology.  If  a  researcher 
is  trying  to  learn  about  a  business  organization,  questions  to  ask  might  include,  “How  would 
you  refer  to  new  trainees  in  your  organization?”  or  “What  do  you  call  people  who  do  that  kind 
of  work?”  As  another  example,  if  interviewing  a  soldier,  one  might  ask,  “What  did  you  call  the 
place  where  you  ate  when  you  were  deployed?” 
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Structural  Questions  Result  In  a  List 

•  Cover  term  questions 

-  What  are  the  different  software  applications  that  are  authorized 

to  be  on  computers  at  this  location? 

•  Included  term/item  questions 

-  I’ve  heard  you  say  that  you’re  allowed  to  have  word-processing 
software  and  spreadsheet  software  on  your  computer.  What  are 
the  database  applications  that  are  also  approved?  What  about 
qualitative  coding  software? 

•  Card  or  list  sorting  questions 

-  Here  are  cards  labeled  with  software  names.  Can  you  divide  them 

by  type  (spreadsheet,  database,  word  processing)?  Which  of 
these  are  authorized  to  be  on  your  machines?  Which  are  easy  to 
operate?  Can  you  divide  them  by  how  frequently  you  use  them? 


RAND 


Structural  questions  are  used  to  build  and  verify  a  domain,  or  a  list  or  structure  of  items 
and  relationships.  Examples  might  include  different  kinds  of  herbs  or  flowers,  reasons  for  join¬ 
ing  the  Army,  the  different  types  of  students  in  a  typical  high  school,  etc.  These  types  of  ques¬ 
tions  help  the  researcher  understand  sets  or  lists  of  things  and  how  they  relate  to  one  another. 
When  asking  these  questions,  it  is  important  to  explain  and  provide  context  and  even  examples 
so  that  respondents  realize  that  the  researcher  is  really  trying  to  build  a  list.  In  the  strict  sense, 
a  researcher  might  use  these  types  of  questions  when  he  or  she  is  trying  to  understand  the 
language  or  semantics  of  those  studied.  However,  they  are  also  useful  techniques  to  flush  out 
knowledge  when  language  is  less  of  a  problematic  issue.  We  discuss  three  types  of  structural 
questions. 

A  cover  term  question  confirms  that  there  exists  a  domain,  or  group  of  items  that  the 
researcher  perceives.  For  example,  there  are  limitations  on  software  applications  that  can  be 
loaded  onto  computers  at  a  secure  location.  Thus,  a  cover  term  question  might  be:  “What  are 
the  different  software  applications  that  are  authorized  to  be  on  computers  at  this  location?” 
Additional  examples  of  cover  term  questions  include:  “What  are  the  different  groups  of  kids  in 
this  high  school?”  “What  are  the  different  reasons  someone  might  join  the  military?”  “What 
are  the  different  kinds  of  training  people  have  problems  with?”  In  these  cases,  the  cover  terms 
are  “authorized  software  applications,”  “groups  of  high  school  kids,”  “different  reasons  to  join 
the  military”  and  “training  people  have  problems  with.” 

An  included  term  or  item  question  attempts  to  put  something  the  researcher  has  heard  in 
context  in  order  to  develop  a  list  further.  “You’ve  explained  that  you  can  have  word-processing 
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and  spreadsheet  software  on  your  computers.  What  other  types  of  software  can  you  have,  such 
as  database  applications  or  qualitative  coding  software?”  Another  example  might  he:  “I  under¬ 
stand  that  ‘jocks’  are  one  group  of  kids  at  this  school.  What  are  other  groups?” 

Card  sorting'll  the  traditional  way  of  asking  respondents  to  group  like  items.  The  researcher 
might  provide  the  respondent  with  actual  cards  with  items  written  on  the  cards,  or  they  might 
do  a  list  sorting  based  on  a  written  list  on  paper  or  a  display  board.  In  these  instances  the 
researcher  might,  for  example,  say,  “Here  are  cards  labeled  with  software  names.  Can  you 
divide  them  by  similar  type,  such  as  all  spreadsheet  applications  together  and  all  word-process¬ 
ing  applications  together?”  That  might  be  followed  by  “Let’s  regroup  them  now  into  a  pile  of 
approved  software  and  a  pile  with  unapproved  software.”  And  when  those  piles  were  complete, 
the  researcher  might  say,  “Please  separate  the  cards  into  a  pile  of  software  applications  that 
would  be  useful  to  you,  regardless  of  whether  they  are  approved,  and  a  pile  of  applications  that 
are  not  necessarily  useful  to  you,  regardless  of  approval  status.” 

Another  example  of  a  card-sorting  exercise  might  involve  job  types  in  an  organization. 
The  researcher  might  say,  “Here  is  a  list  of  all  the  job  types  I’ve  heard  you  explain  in  your  orga¬ 
nization.  Can  you  make  three  piles  based  on  the  prestige  of  these  job  types,  so  that  one  pile 
is  the  jobs  perceived  to  be  the  most  prestigious,  one  is  the  least  prestigious,  and  one  is  in  the 
middle?”  When  that  sort  was  complete,  the  researcher  might  ask,  “Can  you  sort  these  based 
on  the  amount  of  education  required  for  each  job  type?”  That  might  be  followed,  for  example, 
with  “Please  sort  the  job  types  based  on  whether  the  type  is  predominantly  filled  by  women, 
by  men,  or  includes  a  high  proportion  of  both  men  and  women.” 

A  key  difference  between  descriptive  and  structural  questions  is  that  descriptive  questions 
are  process  or  “how”  questions,  and  structural  questions  are  list  questions.  A  structural  ques¬ 
tion,  such  as  “What  experiences  influenced  your  decision  to  join  the  military?”  or  “What  are 
the  reasons  you  joined  the  military?”  will  provide  a  list  of  items,  but  probably  no  sense  of  priori¬ 
tization  or  order  of  events.  “How  did  you  decide  to  join  the  military?”  will  result  in  a  different 
kind  of  answer.  This  is  one  of  the  most  important  distinctions  about  different  question  types. 
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Contrast  Questions  Manipulate  a  List 

•  Contrast  verifications 

-  So  you  are  permitted  to  have  Microsoft  Word  and  PowerPoint, 
but  not  iThink,  Inspiration,  or  NVivo? 

•  Directed  contrasts 

-  So  you  have  Word,  PowerPoint,  and  Excel  on  your  machine. 
What  other  software  do  you  have? 

-  So  you  get  extra  pay  for  being  a  military  pilot.  What  other  kinds 
of  occupations  have  special  pays? 

•  Dyadic  and  triadic  contrasts 

•  Set  sorting 

•  Twenty  questions 

•  Rating  questions 

RAND 


Contrast  questions  help  differentiate  between  items  on  a  list  that  the  researcher  has  already 
obtained. 

Contrast  verification  questions  are  relatively  common.  These  conhrm  that  items  are  dif¬ 
ferent  from  one  another  and  provide  some  conhrmation  of  at  least  one  way  in  which  they  are 
different.  Examples  might  include  “So  you  are  permitted  to  have  Microsoft  Word  and  Micro¬ 
soft  PowerPoint  on  your  computer,  but  you  are  not  authorized  to  have  iThink,  Inspiration,  or 
NVivo?”  Or,  as  another  example,  “So  the  commander  of  a  unit  has  to  approve  voluntary  leave, 
but  not  convalescent  leave?” 

Directed  contrast  questions  focus  on  one  characteristic  of  an  item  and  asks  what  other 
items  share  that  characteristic.  For  example,  “You’ve  told  me  that  you  have  Microsoft  Word, 
PowerPoint,  and  Excel  on  your  desktop  computer.  What  other  software  applications  do  you 
have?”  Another  example  might  be,  “So  you  get  extra  pay  for  being  a  military  pilot.  What  other 
kinds  of  occupations  have  special  pays?” 

Dyadic  and  triadic  contrasts  ask  the  respondent  to  either  suggest  the  differences  between 
two  items  or  terms,  or  to  pick  the  one  that  does  not  fit  with  the  other  two.  There  are  often 
multiple  answers,  and  the  general  point  is  to  understand  how  people  think  about  these  things. 
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One  example  from  a  healthcare  study  of  parents  might  use  the  three  ailments  of  chicken  pox, 
measles,  and  croup,  and  ask  parents  to  say  how  they  are  alike  or  different.  Some  might  say 
chicken  pox  and  measles  are  both  skin-related.  Others  might  suggest  those  two  are  similar 
because  immunizations  exist  for  both  of  them.  Some  might  say  that  measles  and  croup  are 
more  serious,  requiring  a  doctor’s  care.  The  same  study  might  also  provide  another  triad  for 
contrast:  chicken  pox,  bronchitis,  croup.  In  this  case,  some  respondents  might  say  that  bron¬ 
chitis  and  croup  are  both  breathing  problems.  Other  respondents  might  say  that  bronchitis  can 
occur  at  any  age,  while  the  other  two  are  predominantly  childhood  sicknesses.  A  doctor  might 
single  out  bronchitis  as  requiring  an  antibiotic. 

Set  sorting  usually  involves  the  use  of  cards,  and  asks  respondents  to  group  the  cards  by 
whatever  categories  or  themes  they  would  like.  The  researcher  then  may  rescramble  the  cards 
and  ask  the  respondent  to  group  them  again  using  different  themes.  This  is  different  from  the 
card/list  sorting  mentioned  in  the  prior  slide,  because  here  the  respondent  sorts  cards  based  on 
their  own  scheme  or  set  of  domains,  and  thus  might  provide  different  sorts  of  information  in 
the  form  of  categories  that  the  researcher  may  not  have  considered. 

Twenty  questions  can  involve  a  relatively  long  research  interaction.  It  consists  of  asking 
the  respondent  to  ask  yes  and  no  questions  of  the  researcher  regarding  a  list  of  items,  until  the 
respondent  can  guess  which  item  on  the  list  the  researcher  is  thinking  of.  This  tests  whether  the 
researcher  fully  understands  the  topic,  and  permits  the  researcher  to  ask  additional  questions. 
For  example,  if  the  topic  were  a  list  of  software,  the  respondent  might  ask,  “Are  you  thinking 
of  a  software  that  would  permit  you  to  run  regressions?”  And  the  researcher  could  ask  what 
kind  of  regressions  would  be  run,  under  what  circumstances  they  would  be  run,  what  kind  of 
data  might  be  used  for  the  regressions.  This  method  works  especially  well  with  younger  respon¬ 
dents,  who  might  be  more  enthused  about  the  playful  or  “gaming”  nature  of  this  interaction. 
It  obviously  works  less  well  with  more  mature  respondents  who  might  be  time-constrained  or 
less  willing  to  engage  creatively  in  the  interaction. 

Rating  questions  establish  the  order  or  values  of  items,  themes,  activities,  etc.  They  might 
include  asking  someone  the  best  and  worst  Christmas  gifts  received,  what  one  does  first  thing 
in  the  morning,  as  the  day  progresses,  and  at  the  close  of  the  day.  They  might  also  include 
asking  the  best,  mediocre,  and  worst  things  about  a  job.  These  questions  can  be  used  in  the 
course  of  a  semi-structured  interview  when  someone  places  a  value  or  order  on  something, 
such  as  commenting  that  a  particular  job  assignment  is  bad.  The  researcher  might  ask  in  that 
instance,  “what  would  a  good  job  assignment  be?” 

This  material  will  cover  data  collection  later,  but  the  reader  should  bear  in  mind  that 
results  of  these  types  of  questions  can  be  quantified  and  entered  into  various  computer  pro¬ 
grams  that  provide  a  numerical  answer  for  an  individual  interview  or  that  provide  an  aggre¬ 
gated  pattern  of  observations  for  a  set  of  interviews. 
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Problem  Questions 

•  Are  double-barreled 

•  Are  leading 

•  Use  double  negatives 

•  Use  unfamiliar  jargon  or  technical  terms 

•  Are  vague 

•  Use  emotional  language 

•  Are  beyond  the  respondent’s  capability  to  answer 
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There  are  several  things  that  the  researcher  should  be  cautious  of  when  writing  questionsd° 

Doubled-barreled  questions  are  questions  that  are  actually  made  up  of  two  different  ques¬ 
tions,  which  may  have  different  answers.  An  example  of  this  might  be  “How  did  you  learn 
about  the  new  leave  policy  and  how  will  it  impact  your  work  life?”  Double-barreled  questions 
are  difficult  for  respondents  to  answer;  respondents  may  answer  just  one  part  of  the  question 
or  may  be  more  likely  to  provide  ambiguous  answers.  Generally,  it  is  best  to  ensure  that  each 
question  is  asked  independently." 

Leading  questions  encourage  a  respondent  to  answer  in  a  certain  way,  and  they  imply  that 
there  is  a  right  or  wrong  answer.  “You  don’t  support  the  new  policies,  do  you?”  is  an  example 
of  a  leading  question. 

Double  negatives  are  confusing — and  they  are  poor  grammar. 

The  researcher  should  make  sure  that  all  respondents  understand  what  they  are  being 
asked.  The  question  might  appear  perfectly  clear  to  the  researcher,  and  it  may  seem  to  capture 
exactly  what  is  intended.  Nonetheless,  respondents  may  interpret  the  question  differently  or, 
even  worse,  be  unable  to  answer  the  question  because  they  do  not  understand  what  is  being 
asked  of  them.  Testing  questions  can  help  to  clear  up  ambiguities. 


Adapted  from  Neuman  (2006),  Arksey  and  Knight  (1999),  and  Fowler  (1995). 

"  While  grand  tour  questions  are  admittedly  double-  or  triple-barreled,  the  emphasis  in  a  grand  tour  question  is  to  initi¬ 
ate  a  respondent  narrative.  Double-barreled  questions  are  most  problematic  when  they  include  two  key  questions  that  each 
need  to  be  answered. 
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Questions  should  use  neutral  language  in  order  to  prevent  biasing  the  responses.  Certain 
words  hold  power  and  can  affect  how  respondents  “hear”  the  question. 

Finally,  interviews  are  not  tests.  Asking  a  series  of  questions  that  are  beyond  the  grasp  of 
the  respondent  can  be  embarrassing  and  frustrating  for  him  or  her.  It  is  also  a  waste  of  time 
and  effort. 
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Probing  During  the  Interview 

•  Probes  should  be  used  whenever: 

—The  interviewer  doesn’t  understand  the 
respondent’s  reply 

-The  question  specifically  indicates  that  you 
should  probe 

-Respondents 

*  Give  the  interviewer  any  reason  to  think  that  they  have 
not  given  a  compiete  report  of  their  thinking 

•  Say  "don't  know"  or  "I  can't  answer  that” 

*  Give  an  answer  that  doesn’t  fit 

•  Seem  to  have  not  understood  the  question 
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Probing  is  a  way  to  stimulate  the  interview.  Interviewers  use  probes  when  they  do  not  under¬ 
stand  what  the  respondent  has  said  and  thus  need  further  clarification.  Sometimes  questions 
specifically  indicate  that  the  interviewer  should  probe.  In  semi-structured  interviews,  the  inter¬ 
viewer  is  sometimes  asked  to  follow  up  on  an  issue  if  the  topic  does  not  come  up  in  the  respon¬ 
dent’s  response  to  the  initial  question.  For  instance,  on  a  recent  project  respondents  were  asked 
about  physical  aspects  of  having  a  particular  disease.  The  question  was  worded  as  “Tell  me 
about  the  physical  problems  you  have  experienced  due  to  this  disease.”  The  researchers  sus¬ 
pected  that  sleep  disturbances  were  a  large  problem  for  individuals  with  the  disease  and  wanted 
to  make  sure  to  capture  respondents’  experiences  with  that  issue.  So  if  respondents  did  not 
offer  any  indication  of  sleep  disturbances  as  a  problem  in  response  to  the  question,  the  inter¬ 
viewers  probed  with  “What  about  problems  with  sleeping?” 

Interviewers  should  also  probe  when  they  think  that  the  respondent  has  not  told  them 
everything  they  can;  the  answer  provided  is  a  “non-answer,”  in  that  it  does  not  answer  the 
question;  if  the  interviewer  thinks  the  respondent  has  not  understood  the  question;  or  if  the 
respondent  says  “I  don’t  know.”  Sometimes  “I  don’t  know”  really  means  “I  need  more  time  to 
think  about  this”  or  “I  don’t  really  understand  what  you  are  asking.” 

In  deciding  whether  or  not  to  use  probes,  interviewers  always  need  to  listen  very  atten¬ 
tively  to  the  respondent’s  answer  in  order  to  determine  whether  the  answer  is  clear  and  com¬ 
plete.  Interviewers  need  to  be  familiar  with  the  purpose  of  the  question  to  know  when  the 
answer  is  sufficient.'^ 


Adapted  from  RAND  SRG  internal  training  materials  and  best  practices  documentation. 
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Some  Standard  Probes 


FOR  CLARITY/SPECIFICITY 

•  Can  you  be  more  specific? 

•  Can  you  tell  me  more  about 
that? 

•  What  is  your  best 
estimate? 

•  What  do  you  think? 

•  Which  would  be  closer? 

•  Which  answer  comes 
closest  to  how  you  feel/ 
think? 

•  If  you  had  to  pick  one 
answer,  what  would  you 
choose? 


FOR  COMPLETENESS: 

•  Anything  else? 

•  Tell  me  more. 

OTHER  PROBING  TECHNIQUES: 

*  Repeat  the  question 

*  Echo  their  response 

•  Pause  a  second 

*  Baiting 
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Probes  elicit  additional  information  or  clarify  responses.  These  standard  probes  are  neutral  and 
elicit  further  information  without  biasing  the  respondent’s  answer. 

Most  of  the  probes  listed  on  this  slide  are  self-explanatory,  but  there  are  also  other  prob¬ 
ing  techniques. 

The  “tell  me  more”  probe  comes  naturally  to  some  interviewers,  who  frequently  ask,  “Can 
you  tell  me  more  about  that?”  “Why  do  you  feel  that  way?”  “Why  do  you  say  that?”  etc.  The 
correct  tonal  inflection  can  make  the  difference  between  sounding  inquiring  and  sounding 
defensive. 

One  tactic  is  simply  to  repeat  the  question,  giving  the  respondent  more  time  to  think 
about  the  answer. 

An  echo  probe  repeats  the  last  thing  the  respondent  said.  This  is  sometimes  useful  for  lack 
of  a  better  response  when  the  respondent  shares  something  personally  painful,  such  as  “My 
father  was  an  alcoholic.”  Sometimes  repeating,  “Your  father  was  an  alcoholic,”  without  any 
tone  of  judgment,  will  lead  the  respondent  to  further  discussion.  Or,  in  a  different  example, 
the  interviewer  might  repeat,  “So  then  you  have  to  complete  Ranger  school”  and  add,  “then 
what  do  you  have  to  do?”  There  is  some  risk  of  sounding  like  a  nuisance  if  this  probe  is  used 
too  frequently. 

The  interviewer  can  also  bait  the  respondent.  This  probe  method  is  sometimes  used  by 
attributing  controversial  views  to  other,  unnamed,  individuals.  One  example  is  “I’ve  heard 


Adapted  from  RAND  SRG  internal  training  materials  and  best  practices  documentation. 
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some  people  assert  that  the  company’s  new  vacation  policy  is  unfair  to  junior  employees.”  This 
prohe  method  prevents  the  researcher  from  appearing  to  have  a  position  on  a  controversial 
issue,  and  sometimes  promotes  heated  discussion.  This  is  also  useful  when  there  is  a  “secret” 
that  everyone  knows  hut  no  one  is  willing  to  mention  hrst.  In  the  ofhcial  world,  this  might  he 
a  policy  change  that  is  not  yet  official.  Baiting  can  also  he  used  to  imply  that  the  researcher 
already  knows  something  that,  in  fact,  they  know  very  little  about.  This  can  encourage  a 
respondent  to  conhrm  and  share  information,  because  he  or  she  is  not  divulging  “secrets”  if  the 
researcher  already  knows  (Bernard,  2002). 

There  are  a  couple  of  potential  problems  with  probes  for  which  the  researcher  needs  to  be 
prepared.  The  first  is  the  very  real  likelihood  that  “I  don’t  know”  does  mean  that  the  person 
doesn’t  know.  In  these  cases,  probing  may  compel  the  respondent  to  say  things  that  are  not 
true  or  that  he  or  she  does  not  really  feel  in  order  to  please  the  researcher.  The  second  potential 
problem  is  an  ethical  one.  If  the  researcher  has  probed  an  individual  to  speak  freely,  and  the 
respondent  proceeds  to  share  extremely  personal  and  painful  information,  then  the  researcher 
has  to  consider  the  ethical  issues  involved  in  using  that  information.  Is  the  researcher  free  to 
use  the  information,  given  that  the  respondent  may  regret  his  or  her  comments  later?  What  if 
the  respondent  experiences  personal  pain  as  a  result  of  sharing  with  the  researcher?  Is  it  appro¬ 
priate  for  the  researcher  to  pack  up  the  notes  and  leave  a  distraught  person  behind? 

In  some  instances,  interviewers  are  required  to  provide  the  reference  numbers  for  coun¬ 
seling  or  other  support  services  available  to  respondents,  and  some  projects  will  even  desig¬ 
nate  a  clinician  to  respond  in  situations  where  a  series  of  questions  leaves  a  respondent  very 
upset.  These  extreme  cases  aside,  interviewers  have,  at  a  minimum,  such  ethical  responsibilities 
as  maintaining  confidentiality  guidelines,  providing  respondents  with  adequate  information 
so  that  they  may  make  an  informed  decision  regarding  whether  or  not  to  participate  in  the 
research,  and  allowing  respondents  the  option  of  declining  to  answer  questions  with  which 
they  are  uncomfortable. 
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Exercise  1:  Interview  Probes 

*  Review  sample  research  questions 

*  List  all  the  possible  types  of  participants  for  each 
research  question 

*  Identify  probes  for  the  highlighted  research 
question 
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In  Exercise  1,  you  will  review  sample  research  questions,  consider  the  appropriate  types  of  par¬ 
ticipants  to  answer  the  research  question,  and  craft  appropriate  prohes.  The  exercise  is  provided 
on  pp.  117-118. 
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Overview 

•  Introduction  to  data  collection  methods 

*  Applications  from  the  real  world 

*  Semi-structured  interviews 

-  Frame  the  research 
—  Sampling 

—  Designing  questions  and  probes 

-  Developing  the  protocol 

—  Preparing  for  the  interview 
—  Conducting  the  interview 

-  Capturing  the  data 

•  Focus  groups 

*  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 
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This  portion  of  the  course  focuses  on  developing  a  protocol. 
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Protocols  Are  Necessary 

*  Clarify  questions  and  identify  probes 

*  Ensure  that  interviews  are  consistent 

-  Across  interviewers 

-  Across  respondents 

*  Prioritize  research  questions 

*  Intended  only  for  interviewer  use;  not  to  be 
shared  with  interviewee 

-  Send  topics,  not  protocol 
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Protocols  allow  researchers  to  structure  the  interview.  The  process  of  developing  a  protocol 
includes  the  formation  of  questions  and  prohes  and  thus  compels  the  researcher  to  clarify  and 
prioritize  the  information  wanted  from  each  interview. 

When  researchers  collect  data,  they  want  to  minimize  the  effects  of  different  actors  on 
the  data  collection  (interviewer  bias).  It  is  important  that  interviews  are  administered  consis¬ 
tently  both  across  different  interviewers  and  across  different  respondents.  If  a  research  effort 
includes  multiple  interviews,  it  is  important  that  the  first  interview  covers  the  same  topics  as 
the  very  last  interview  conducted.  If  there  are  multiple  interviewers,  it  is  important  that  each 
interviewer  asks  the  same  questions  in  the  same  way.  Otherwise,  the  data  are  not  comparable. 

Additionally,  interviews  are  time-constrained,  so  the  protocol  guides  the  researcher  to 
prioritize  the  research  questions  and  to  understand  which  questions  are  “must  ask”  and  which 
are  secondary. 

One  important  note  is  that  the  protocol  is  for  the  interviewers’  use,  not  the  respondents’. 
If  the  respondent  wants  to  see  a  list  of  questions  before  the  interview,  the  interviewer  should 
send  a  topic  list  rather  than  the  actual  protocol.  If  the  respondent  receives  the  protocol  prior  to 
the  session,  they  may  provide  only  “official  answers”  or  answers  prepared  by  a  staff  of  assistants. 
Also,  the  interview  data  will  be  inconsistent  and  will  not  be  comparable  if  one  respondent 
received  the  protocol  in  advance  while  another  did  not. 
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Designing  Protocoi  Schemes  for 
Semi-Structured  Interviews 

•  Funnel  protocols 

-  Broad  questions  leading  to  more  focused  questions 

-  Builds  rapport  (useful  when  topic  is  sensitive) 

*  Inverted  funnels 

-  Narrow  questions  leading  to  broad  discussion 
—  Places  interviewees  in  the  context  of  the  topic 

—  Let  interviewees  become  comfortable  before  they  talk  freely 

•  Tunnel  method 

-  Avoids  broad  questions 

—  Appropriate  when  time  is  limited 

*  Quintamensional  method 

-  Assesses  the  intensity  of  opinions 
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Having  discussed  the  type  of  questions,  it  is  useful  to  consider  the  sequencing  theme  of  the 
protocol  in  which  different  questions  are  posed.'"* 

Funnel protocols  employ  hroad  questions,  such  as  grand  tour  questions,  before  asking  more 
pointed  questions.  This  is  appropriate  for  sensitive  topics.  It  would  he  difficult,  for  example, 
for  a  researcher  to  introduce  oneself  and  then  immediately  ask,  “How  serious  is  your  drinking 
prohlem?” 

Inverted  funnels  begin  with  closed  questions,  often  background  questions,  and  gradually 
build  to  more  open-ended  questions.  These  may  gather  important  background  and  also  permit 
the  respondent  time  to  become  comfortable  with  the  interview  before  being  asked  more  sensi¬ 
tive  or  broad  questions. 

The  tunnel  method  involves  a  series  of  similar  questions  where  the  respondent  may  have  to 
rate  things,  but  where  the  approximate  depth  of  the  questions  are  similar  throughout.  Tunnel 
protocols  avoid  broad  questions  and  are  most  appropriate  when  the  researcher  has  only  limited 
time  with  a  respondent. 

A  quintamensional  protocol  determines  the  intensity  of  a  respondent’s  opinions  and  atti¬ 
tudes  with  hve  steps  that  assess  (1)  the  degree  of  awareness  of  an  issue,  (2)  uninfluenced  atti¬ 
tudes,  (3)  specific  attitudes,  (4)  reasons  for  these  attitudes,  and  (5)  intensity  of  these  attitudes. 
This  method  was  developed  by  Gallup  in  1947,  but  Stewart  and  Shamdasani  (1990)  provide  a 
useful  example,  as  follows: 


14 


Adapted  from  Stewart  and  Shamdasani  (1990). 
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1 .  “Tell  me,  what  do  you  know  about  current  methods  of  disposing  of  hazardous  chemi¬ 
cal  waste?” 

2.  “What,  if  any,  are  the  factors  that  contribute  to  the  growing  stockpile  of  hazardous 
chemical  waste?” 

3.  “Do  you  approve  or  disapprove  of  these  methods  of  disposing  of  hazardous  chemical 
wastes?” 

4.  “Why  do  you  feel  this  way?” 

5.  “How  strongly  do  you  feel  about  this?  Strongly,  very  strongly,  something  that  you  would 
not  change  your  mind  on?” 
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What  Protocol  Looks  Like  -  Formats 

Protocol  should  contain: 

*  Introduction 

*  Ground  rules 

•  Questions  and  probes 

-  Questions  should  be  grouped  by  topic 

—  Decide  ahead  of  time  how  many  minutes  you  want  to 
spend  on  each  topic  area 

—  Think  carefully  about  the  number  of  questions 

-  Think  carefully  about  appropriate  probes 

•  Thank  you  and  next  steps 

RAND 


Protocols  begin  with  an  introduction.  The  researcher  generally  introduces  his  or  herself,  their 
organization,  the  purpose  of  the  research,  and  the  reason  that  the  respondents  have  been  asked 
to  participate  in  the  interview. 

Next,  the  researcher  lays  the  ground  rules.  This  includes  the  length  of  time  of  the  inter¬ 
view,  any  assurances  the  researcher  has  to  safeguard  information,  and  information  about  the 
types  of  reporting  that  will  come  from  the  data. 

The  questions  come  next.  A  protocol  generally  groups  questions  by  topic  since  it  makes 
sense  for  the  flow  of  conversation.  The  actual  conversation  might  not  follow  the  flow  specifled, 
but  the  protocol  will  help  the  researcher  keep  track  of  what  has  been  answered  and  what  still 
needs  to  be  covered.  Specific  probes  may  also  be  outlined  in  the  protocol. 

The  researcher  should  think  carefully  about  how  much  time  is  to  be  allotted  to  the  inter¬ 
view  and  how  many  questions  can  be  covered  in  that  time.  If  an  interview  is  30  minutes  long, 
can  it  really  cover  15  questions?  After  an  introduction,  there  would  be  less  than  2  minutes 
remaining  per  question,  which  would  not  permit  much  of  a  discussion.  Again,  it  is  important 
to  decide  how  much  time  will  be  devoted  to  each  topic.  If  different  interviewers  spend  signifi¬ 
cantly  different  amounts  of  time  on  each  interview  topic,  then  the  results  may  include  missing 
data  and  substantially  different  interviews. 


The  content  of  the  introduction  may  also  be  guided  by  the  informed  consent  requirements  of  an  organization’s  Institu- 
tional  Review  Board. 
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Finally,  at  the  end  of  the  interview  the  researcher  should  take  time  to  thank  the  respon¬ 
dent  and  indicate  the  next  steps  in  the  process.  Does  the  interviewer  need  to  send  the  respon¬ 
dent  a  follow-up  document?  Has  the  respondent  promised  to  send  something  to  the  inter¬ 
viewer?  This  is  the  time  to  have  that  discussion. 
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Exercise  2:  Interview  Protocol 

•  Review  sample  protocol 

•  Identify  the  sections 

•  Discuss  problems 
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In  Exercise  2,  you  will  review  a  sample  interview  protocol  and  discuss  any  problems  with  it. 
The  exercise  is  provided  on  pp.  119-120. 
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Testing  the  Protocol 

•  Are  questions  understandable  and  appropriate? 

•  Are  responses  what  you  expected? 

•  Is  the  flow  correct? 

•  Do  other  issues  arise  that  you  should  include? 

•  Review  for  mis-administered  questions 

-  Terminology  or  pronunciation 

-  Correctly  used  probes 
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Any  protocol  instrument  designed  for  data  collection  should  be  tested.  Sometimes  researchers 
use  a  detailed  approach  to  testing  instruments,  such  as  cognitive  interviewing,  where  there  is 
the  opportunity  to  delve  into  the  meaning  of  a  single  question,  or  even  a  single  word,  with  a 
respondent.  At  a  minimum,  researchers  should  test  the  instrument  on  a  convenience  sample  of 
people  who  will  be  similar  to  the  respondents  who  will  participate  in  the  study. 

When  testing  questions,  researchers  should  look  for  question  problems,  such  as  vague 
answers,  unclear  terminology,  or  questions  that  are  inappropriate  for  the  audience.  Are  the 
answers  addressing  the  questions  the  researcher  thinks  they  are  asking?  Does  the  flow  of  topics 
and  questions  seem  sensible  and  natural? 

This  testing  should  ensure  that  the  terminology  of  the  questions  will  be  properly  under¬ 
stood  by  the  respondent.  If  there  are  multiple  interviewers,  researchers  should  consider  whether 
terms  or  words  could  be  mispronounced  and  whether  the  probes  will  be  used  correctly. 
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Overview 

•  Introduction  to  data  collection  methods 
'Applications  from  the  real  world 

•  Semi -structured  interviews 

—  Frame  the  research 
—  Sampling 

-  Designing  questions  and  probes 

-  Developing  the  protocol 

—  Preparing  for  the  interview 
—  Conducting  the  interview 

-  Capturing  the  data 

•  Focus  groups 

■  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 
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This  portion  of  the  course  discusses  the  preparation  for  the  interview.'*’ 


**’  This  section  on  preparing  for  the  interview  is  adapted  from  RAND  SRG  internal  training  materials  and  best  prac¬ 
tices  documentation.  Readers  who  desire  more  information  should  see,  for  example,  Arksey  and  Knight  (1999)  and 
Fowler  (2002). 
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The  Interview  Team 

•  Interviewers  must  be  able  to: 

-  Gain  cooperation 

-  Listen 

—  Be  neutrai 

—  Maintain  confidentiaiity 

•  Note-takers  must  be  able  to: 

-  Record  accurately 

—  Note  subtleties,  nonverbal  behavior 
—  Understand  their  role  in  the  session 
—  Understand  when  clarification  is  needed 

-  Be  cost-effective 

RAND 


An  important,  and  often  neglected,  aspect  of  interview-based  research  is  the  proper  selection, 
training,  and  follow-up  with  the  data  collection  team. 

The  first  part  of  the  team  is  the  interviewer.  Not  everyone  makes  a  good  interviewer. 
Interviewers  need  to  be  able  to  get  a  respondent  to  participate  and  stay  engaged  in  the  interac¬ 
tion.  One  might  think  of  good  interviewers  as  those  able  to  project  “friendly  professionalism” 
in  order  to  build  rapport. 

Interviewers  must  be  able  to  listen  closely  to  the  respondent  in  order  to  determine  whether 
the  questions  are  being  answered  and  determine  when  follow-up  is  needed.  This  is  not  always 
easy  after  an  interviewer  has  conducted  many  interviews  and  may  think  he  or  she  has  already 
heard  all  the  possible  answers. 

Interviewers  must  also  stay  neutral,  no  matter  what  he  or  she  thinks  or  hears.  Respon¬ 
dents  can  offer  surprising  responses,  be  controversial,  and  even  try  to  provoke  the  interviewers. 
Good  interviewers  remain  unfazed  and  do  not  offer  their  own  opinions. 

Interviewers  must  also  be  able  to  maintain  the  confidentiality  of  the  respondents. 

Another  key  member  of  the  team  is  the  note-taker.  It  is  best  practice  to  have  a  note-taker 
as  well  as  an  interviewer.  Taking  detailed  notes  and  conducting  an  interview  at  the  same 
time  is  virtually  impossible.  That  is  not  to  say  that  an  interviewer  does  not  take  any  notes; 
each  interviewer  should  be  taking  some  notes  during  the  interaction.  However,  the  interviewer 
should  leave  detailed  notes  to  the  note-taker,  as  interviewers  cannot  capture  the  detail  while 
simultaneously  paying  attention  to  the  respondent  and  making  sure  that  the  protocol  has  been 
followed.  Considerable  data  can  be  lost  if  the  interviewer  is  required  to  play  all  roles. 
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As  with  interviewers,  not  everyone  is  well  suited  to  be  a  note-taker.  A  note-taker  must  be 
able  to  take  highly  detailed  and  accurate  notes  during  the  interview.  Note-takers  need  to  be 
able  to  write  quickly.  Note-takers  are  also  responsible  for  noting  any  nonverbal  behavior  (which 
can  be  very  telling),  any  subtleties  in  tone,  and  also  the  “background”  of  the  interview,  that  is, 
what’s  going  on  around  the  interview  that  might  have  affected  the  discussion.  For  instance,  are 
there  people  coming  in  and  out  of  the  room? 

Note-takers  need  to  understand  their  role;  the  note-taker  is  not  conducting  the  interview, 
but  he  or  she  may  need  to  clarify  information  as  necessary  to  ensure  that  the  notes  are  cor¬ 
rect.  Thus,  a  note-taker  needs  to  understand  when  interjecting  is  appropriate.  The  interviewer 
can  help  with  this.  Good  interviewers  often  turn  to  the  note-taker  after  a  topic  is  completed 
or  if  the  respondent  has  just  provided  a  long  or  complicated  explanation,  and  ask,  for  example, 
“Robert,  did  you  get  that,  or  do  you  need  to  clarify  anything?” 

One  last  thought  is  that  researchers  should  want  the  note-takers  to  be  cost-effective.  There 
is  no  need  to  have  the  most  senior  person  on  the  research  project  do  the  note-taking.  Instead, 
one  should  think  about  using  a  junior  colleague  as  a  note-taker,  perhaps  someone  who  would 
benefit  from  exposure  to  a  particular  subject  or  area  of  research.  Note-taking  can  also  be  a 
good  training  opportunity  for  a  prospective  interviewer. 
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Training  and  Foiiow-Up  with  interview  Team 

*  Ensure  that  your  team  is  trained 

—  Interviewing  skills 
-Note-taking  skills  and  conventions 
-Project  specific  training 

*  Ongoing  interaction 

—  Interviewer  debriefings 
-  Reports  to  team 
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Researchers  should  ensure  that  the  data  collection  team  not  only  has  training  on  basic  inter¬ 
viewing  skills,  hut  also  training  on  how  to  take  good  notes  and  on  the  note-taking  conven¬ 
tions  that  the  project  team  has  decided  upon.  The  person  who  does  the  analysis  is  not  always 
the  person  who  has  conducted  the  interview  or  taken  the  notes.  When  someone  sits  down  to 
analyze  50  sets  of  interview  notes,  it  is  important  that  all  note-takers  meant  the  same  thing  hy 
those  symbols  and  notations.  Notes  should  be  able  to  stand  the  test  of  time  and  be  understand¬ 
able  even  if  a  researcher  resumes  using  them  two  years  later. 

The  team  should  also  have  project-specific  training.  What  is  the  purpose  of  the  study, 
what  are  the  procedures  for  contacting  respondents,  what  is  the  intention  of  the  questions 
being  asked?  This  will  allow  interviewers  to  know  how  to  address  potential  questions  from  the 
respondents,  and  know  when  they  have  collected  the  proper  data. 

General  interviewer  training  should  cover  the  basics  of  the  research  process  and  general 
interviewing  skills,  such  as  how  to  probe  and  how  to  avoid  refusals,  and  other  topics.  Then, 
when  interviewers  are  assigned  to  actual  projects,  they  go  through  several  additional  days  of 
project-specific  training.  Project-specific  training  includes  a  complete  review  of  the  data  collec¬ 
tion  instruments,  overviews  of  the  project  purpose,  and  training  on  how  to  address  questions 
from  respondents  about  the  research. 

Interaction  should  be  ongoing.  The  lead  researcher  should  not  simply  train  interviewers 
and  note-takers  and  send  them  off  to  conduct  hundreds  of  interviews.  Instead,  project  teams 
should  meet  again  after  the  first  few  interviews  and  debrief  This  is  when  researchers  can  deter¬ 
mine  whether  questions  are  working,  whether  there  are  significant  and  unexpected  differences 


60  Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups 


across  the  population,  or  whether  there  are  additional  unanticipated  issues  or  topics.  If  a  project 
team  meets  early  in  the  interview  process,  it  is  possible  for  the  researchers  to  make  critical  fixes 
before  it  is  too  late.  Also,  periodic  meetings  allow  the  research  team  to  assess  whether  there 
has  been  divergence  among  the  team  in  how  the  interviews  have  been  conducted,  to  consider 
whether  there  are  any  new  issues  or  problems,  and  to  keep  everything  on  track. 

The  lead  researcher  should  also  request  updates  on  how  many  interviews  are  being  com¬ 
pleted,  how  long  the  interviews  are  taking,  and  other  logistical  or  administrative  consider¬ 
ations.  This  can  be  essential  in  keeping  projects  on  track  and  on  schedule. 
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Gaining  interview  Access 

•  Mode  of  contact 

*  Consider  whether  you  can  contact  them 
directly 

-  Gatekeepers 

-  Competing  demands  for  their  time 

-  Authorized  use  of  time 
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There  are  many  factors  to  consider  when  deciding  how  to  contact  the  interview  respondents. 
For  research  done  within  the  workplace,  such  as  for  official  lessons-learned  activities,  many  of 
these  access  issues  are  much  easier. 

If  the  researchers  do  not  have  a  professional  relationship  with  the  respondents,  then  there 
are  more  decisions  and  more  difficult  challenges.  Will  the  initial  contact  be  by  phone,  by  email, 
or  by  letter?  In  some  instances,  the  interviewer  might  be  able  to  contact  the  respondent  directly. 
In  other  cases,  the  interviewer  will  need  to  send  information  in  advance  or  have  someone  else 
make  the  initial  contact. 

Gatekeepers  are  individuals  who  control  access  to  the  respondent.  Researchers  almost 
always  run  into  gatekeepers  when  trying  to  interview  those  who  might  be  considered  high-status 
people,  such  as  doctors  or  senior  government  officials,  but  gatekeepers  can  also  be  individuals 
such  as  spouses.  It  is  helpful  if  the  interviewer  can  develop  a  rapport  with  the  gatekeeper,  as  the 
gatekeeper  can  help  the  interviewer  access  the  respondent. 

When  contacting  the  respondents,  researchers  need  to  consider  the  respondents’  sched¬ 
ules  and  demands  on  their  time.  Sometimes  interviewers  need  to  be  creative  about  how  and 
when  they  contact  people.  For  instance,  sending  FedEx  packages  rather  than  letters  can  be 
helpful,  as  FedEx  packages  receive  more  attention.  In  addition,  calling  before  or  after  standard 
office  hours,  when  receptionists  or  secretaries  are  likely  to  have  gone  home,  but  respondents 
may  still  be  in  the  office,  can  also  be  profitable. 

For  some  projects,  it  makes  sense  to  have  the  project  sponsor  send  the  potential  respon¬ 
dents  some  advance  notice — either  a  letter  or  an  email — that  explains  the  context  of  the 
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research  and  why  it  is  important  that  the  individual  participate.  This  is  especially  important 
when  individuals  may  not  feel  authorized  to  use  ofEcial  time  to  participate  in  an  interview 
unless  they  have  received  ofEcial  approval  or  tasking.  Such  an  introduction  may  also  he  very 
important  to  access  more  senior  people,  if  someone  senior  to  the  respondent,  or  even  parallel 
to  the  respondent,  in  the  organization  has  introduced  the  research.  Almost  without  exception, 
people  are  more  likely  to  he  willing  to  participate  if  the  researcher  mentions  who  suggested 
that  he  or  she  talk  to  the  respondent,  and  gatekeepers  may  not  permit  one  access  unless  the 
researcher  has  formal  entree. 
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Gaining  the  Respondent’s  Cooperation 

*  Introduction 

—  Explain  who  you  are,  what  you  are  doing 

—  Have  interviews  have  been  approved/ 
authorized? 

—Why  are  their  opinions  important? 

—  Is  anonymity  and  confidentiality  offered? 

*  Be  prepared  for  questions 

*  Take  the  initiative 
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Depending  on  whom  the  interviewer  is  contacting,  it  may  he  beneficial  to  send  an  advance 
email  or  letter  with  a  brief  project  description.  Regardless  of  how  the  respondent  is  initially 
contacted,  the  researcher  should  inform  him  or  her  of  the  researcher’s  organizational  affiliation 
and  the  purpose  and  intent  of  the  project.'^ 

Potential  respondents  often  will  have  other  questions  that  need  to  be  addressed  before 
they  are  willing  to  participate.  Interviewers  need  to  sound  knowledgeable  about  the  study  and 
able  to  address  these  questions  without  stumbling  and  without  “pregnant  pauses.”  Researchers 
should  provide  interviewers  with  a  list  of  questions  that  might  be  asked,  and  the  best  answers  to 
provide  to  respondents.  By  doing  so,  every  respondent  will  receive  the  same  description  about 
the  study  and  the  same  explanation  about  why  and  how  he  or  she  was  selected  as  a  respondent. 
This  is  another  way  to  ensure  consistency  across  interviews. 

Interviewers  should  listen  to  the  respondent’s  concerns  and  address  those  concerns  specifi¬ 
cally.  Interviewers  should  also  be  proactive:  If  an  interviewer  hears  hesitancy  on  the  part  of  the 
respondent,  the  interviewer  should  anticipate  and  try  to  address  his  or  her  concerns.  Interview¬ 
ers  might  also  offer  to  call  at  another  time,  offer  to  meet  late  in  the  day  or  early  in  the  morning, 
or  consider  whether  the  interview  can  be  completed  by  telephone  (if  this  is  an  option  approved 
for  the  research  project). 


The  content  of  the  introduction  may  be  guided  by  the  infotmed  consent  tequitements  of  the  otganization’s  Institutional 
Review  Boatd.  In  the  case  of  intetnal  otganization  teseatch  (such  as  lessons-leatned  activities),  the  tespondent  may  be  awate 
of  the  teseatchet’s  otganizational  affiliation.  Nonetheless,  a  cleatly  wtitten  statement  about  the  putpose  and  intent  of  the 
study  will  still  be  beneficial. 
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Overcoming  Common  Barriers 

*  “I’m  not  interested” 

-  Explain  purpose  of  research  and  remind  them  it’s  an  opportunity 
for  their  opinions  to  be  heard 

*  “I’m  too  busy.  How  long  will  this  take?” 

-  Be  honest  about  time  and  suggest  you  can  schedule  for  a  more 
convenient  time 

*  Fear  of  being  inadequate 

-  Provide  reassurance  that  we  want  their  opinion,  that  there  are  no 
right  or  wrong  answers 

*  Have  a  negative  reaction  to  research 

-  Reiterate  why  the  research  is  important,  provide  information  on 
legitimacy  of  the  research 

*  Confidentiality  concerns,  questions  too  personal 

-  Explain  that  answers  will  be  combined  with  answers  from  others 
and  presented  in  aggregate  form;  no  names  will  be  reported 
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Barriers  to  the  interview  existing  in  the  respondent’s  mind  need  to  he  overcome.  The  inter¬ 
viewer  should  take  the  initiative  to  address  those  concerns. 

Knowing  how  to  address  concerns  will  help  to  avert  refusals.  Having  brief,  prepared 
answers  provides  enough  information  to  gain  cooperation  without  sacrificing  considerable 
time  with  lengthy  conversations.  Here  is  a  list  of  some  common  concerns  expressed  by  poten¬ 
tial  respondents. 

When  someone  says  they  are  not  interested,  the  interviewer  might  try  letting  the  respon¬ 
dent  know  what  is  interesting  about  the  research  and  why  the  research  is  potentially  important 
to  the  respondent.  They  can  also  remind  the  respondent  that  the  interview  affords  him  or  her 
an  opportunity  to  state  individual  opinions  on  the  topic. 

If  a  potential  respondent  is  too  busy,  the  interviewer  should  try  a  work-around.  The  inter¬ 
viewer  might  let  the  respondent  know  how  much  time  is  really  necessary  for  the  interview  (if 
a  concise  version  can  be  conducted),  offer  alternative  days  or  times,  or  consider  whether  it  is 
permissible  to  break  the  interview  into  smaller  pieces. 

In  some  instances,  the  respondent  may  lack  sufficient  experience  or  the  expertise  to  par¬ 
ticipate.  Sometimes  this  is  actually  the  case,  but  sometimes  the  research  is  designed  to  include 
less-experienced  people.  In  these  instances,  a  response  might  offer,  “It’s  important  for  us  to 
speak  with  people  with  a  range  of  experience  to  get  a  true  picture  of  what  is  happening.” 

Some  people  are  very  skeptical  of  the  research  process.  The  respondent  needs  to  see  the 
survey  as  being  important  and  worthwhile.  In  these  instances,  the  respondent  may  appreciate 
information  on  the  legitimacy  of  the  research  or  on  how  the  findings  will  be  used. 
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If  a  respondent  has  confidentiality  concerns,  then  it  is  helpful  for  the  interviewer  to  reiter¬ 
ate  any  protections  the  project  can  offer  and  to  inform  him  or  her  about  how  the  data  will  he 
presented. 
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Overview 

•  Introduction  to  data  collection  methods 

•  Applications  from  the  real  world 
•Semi-structured  interviews 

—  Frame  the  research 
—  Sampling 

-  Designing  questions  and  probes 
—  Developing  the  protocol 

—  Preparing  for  the  interview 
—  Conducting  the  interview 

-  Capturing  the  data 

•  Focus  groups 

•  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 

RAND 


The  next  portion  of  this  course  focuses  on  how  to  conduct  an  interview.*^ 


This  section  on  conducting  the  interview  is  adapted  from  RAND  SRG  internal  training  materials  and  best  prac¬ 
tices  documentation.  Readers  who  desire  more  information  should  see,  for  example,  Arksey  and  Knight  (1999  and 
Fowler  (2002). 
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Initial  Considerations  for  the  Interview 

*  Selecting  the  location 

*  Choosing  situation-specific  dress 

*  Opening  the  interview 

-  Refer  to  earlier  contacts  with  same  person 

-  Review  history  of  your  organization 

-  indicate  significance  and  potentiai  benefits  of  study 

-  Expiain  interview  process 

-  Expiain  pubiishing  process 

-  Discuss  confidentiality/anonymity/consent  form 

-  Acknowiedge  any  recording 

-  Provide  opportunity  to  ask  questions 

RAND 


The  location  selected  for  the  interview  is  very  important.  Ideally,  interviews  should  be  con¬ 
ducted  in  a  private,  quiet  space  with  no  distractions.  Other  people  coming  in  and  out  of  the 
area  are  both  disruptive  to  the  interview  process  and  also  likely  to  reduce  the  candor  of  the 
respondent. 

How  an  interviewer  dresses  for  the  session  depends  on  the  respondents  and  the  context  of 
the  interview.  The  most  appropriate  dress  is  situation-specific. 

When  the  interviewer  opens  the  interview,  there  are  a  few  “pieces  of  business”  that  need 
to  be  covered.  First,  the  interviewer  should  provide  some  information  about  the  research  orga¬ 
nization.  The  interviewer  should  also  explain  the  interview  process,  including  the  likely  length 
of  the  session,  the  topic  of  the  questions,  and  whether  the  respondent  can  refuse  to  answer  any 
questions.  The  beginning  of  the  interview  is  also  the  appropriate  time  to  explain  the  final  pub¬ 
lication  process,  including  any  public  clearance  or  release  process,  and  whether  the  respondent 
will  have  an  opportunity  to  review  the  report.  Issues  of  confidentiality,  anonymity,  and  consent 
should  be  addressed  at  the  beginning  of  the  session. Tfiis  is  important  even  if  these  issues  were 
discussed  when  the  interview  was  scheduled.  If  the  interviewer  is  audio-  or  video-recording  the 
session,  the  recording  needs  to  be  acknowledged  and  approved  by  the  respondent.  The  inter¬ 
viewer  should  not  assume  that  recording  is  acceptable  and  permitted.  All  interviews  should 
begin  only  after  the  interviewer  has  asked  whether  the  respondent  has  any  questions. 


The  statements  regarding  confidentiality,  anonymity,  and  consent  will  likely  be  guided  by  the  informed  consent  require¬ 
ments  of  the  organization’s  Institutional  Review  Board. 
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Asking  Questions 

•  Be  very  familiar  with  the  questions  in  the  interview 
protocol 

-  Understand  the  intent  of  a  question  -  what  are  you 
asking? 

*  Make  sure  that  you  cover  all  the  material  in  the 
interview  protocol 

•  Listen  carefully  to  the  response 

-  Evaiuate  whether  the  question  has  reaiiy  been  answered 

-  Probe  for  additionai  information  as  needed 

*  Maintain  a  neutral  attitude 

RAND 


As  discussed  previously,  the  interviewer  must  understand  the  question  in  order  to  respond  to 
any  inquiries  from  the  respondent  about  the  question. 

Sometimes  questions  will  he  asked  out  of  order  because  of  the  natural  flow  of  conversa¬ 
tion,  but  the  interviewer  needs  to  make  sure  that  every  question  in  a  topic  section  is  addressed, 
ideally  before  moving  on  to  the  next  topic.  Sometimes  it  is  helpful  to  check  off  questions  as 
they  are  addressed.  Before  closing  the  interview,  the  interviewer  should  review  the  protocol  and 
return  to  questions  that  were  not  covered. 

When  the  respondent  is  answering,  the  interviewer  should  practice  “active  listening,”  lis¬ 
tening  carefully  to  the  respondent  and  evaluating  whether  the  question  has  been  fully  answered 
(Morgan,  2006).  Interviewers  should  dig  deeper  for  more  information  if  necessary.  Finally, 
interviewers  need  to  stay  neutral;  this  includes  being  neutral  in  probes,  and  also  in  verbal  and 
nonverbal  cues. 
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Remaining  Neutral 

•  Ideally,  each  respondent  will  interpret  the  question  in 
the  same  way 

•  It  is  important  to  maintain  neutrality  so  as  not  to 
influence  how  the  respondent  answers  the  question 

-  Do  not  suggest  an  answer 

-  Do  not  assume  answers  ahead  of  time 

-  Do  not  give  your  own  opinions 

-  Do  not  agree  or  disagree  with  the  person’s  comments 

-  Avoid  making  gestures  that  could  indicate  that  you  approve  or 
disapprove  of  a  respondent’s  answer  or  comment 

•  Remember,  there  are  no  “right”  or  “wrong”  answers 
and  you  are  not  judging  the  respondents’  answers 
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In  order  for  two  answers  to  be  comparable,  the  question  should  mean  the  same  thing  to  each 
respondent. 

When  an  interviewer  does  not  maintain  a  neutral  manner,  he  or  she  may  influence  or  bias 
the  data  by  affecting  how  respondents  interpret  a  question. 

Most  respondents  want  to  be  helpful  and  provide  information  that  they  think  will  be 
helpful  to  the  research.  If  a  respondent  is  given  the  impression  that  certain  kinds  of  answers  are 
preferable,  they  may  provide  more  of  the  “preferred”  answers  in  an  effort  to  please  the  inter¬ 
viewer.  Using  probes  that  contain  phrases  like  “this  is  good  information”  is  biasing  because  it 
implies  that  some  responses  provide  good  information  and  other  responses  provide  bad  infor¬ 
mation.  Likewise,  an  interviewer  should  never  give  his  or  her  own  opinion  to  a  respondent,  or 
suggest  an  answer  to  the  respondent,  because  the  respondent  will  make  judgments  based  on 
those  interviewer  comments  and  edit  his  or  her  responses  accordingly. 

Interviewers  should  also  never  assume  how  someone  will  answer.  Instead,  the  interviewer 
should  ask  each  question  and  be  open  to  whatever  response  is  provided.  The  interviewer  should 
not  neglect  to  probe  because  of  an  interpretation  of  what  the  respondent  is  “really  trying  to 
say. 

Verbal  and  nonverbal  behaviors,  even  unconscious  ones,  can  bias  an  interview.  For 
instance,  excessive  nodding  of  the  head  or  using  phrases  such  as  “you  are  right”  might  unin¬ 
tentionally  “steer”  the  respondent  to  answer  in  a  certain  way.  An  interviewer  should  not  limit 
his  or  her  demeanor  and  facial  expressions  to  the  extent  that  the  interviewer  appears  robotic, 
but  the  interviewer  should  try  to  be  aware  of  the  social  cues  being  projected. 
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A  slight  caveat:  Some  nonverbal  behaviors  can  be  used  as  probes  or  to  help  direct  the 
interview.  If  an  interviewer  uses  a  quizzical  look  it  might  “probe”  the  respondent  to  provide 
more  detail  or  give  more  explanation.  Likewise,  if  the  interviewer  suddenly  stops  taking  notes, 
it  can  be  a  clue  to  the  respondent  that  the  information  he  or  she  is  providing  does  not  answer 
the  question. 

The  bottom  line  is,  an  interviewer  should  be  aware  of  his  or  her  own  verbal  and  nonverbal 
behavior  and  should  never  lead  the  respondent  to  believe  there  are  right  or  wrong  answers. 
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Aspects  of  a  Successful  Interview 


*  Giving  and  obtaining 
expianations 

-  Project  explanations 

—  “Native  language” 
explanations 

*  Asking  questions 

*  Asymmetricai  turn-taking 


•  Expressing  interest 

•  Expressing  cuiturai  ignorance 

•  Repeating 

•  Restating  respondent’s  terms 

•  incorporating  respondent’s 
terms 

•  Creating  hypothetical 
situations 


Understand  the  answers! 
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Having  discussed  the  aspects  of  successful  interviewing,  here  is  a  quick  summary  that  makes 
clear  the  difference  between  an  interview  and  a  typical  conversation. 

By  the  end  of  a  successful  interview,  the  interviewer  has  accomplished  many  things.  First 
and  foremost,  the  interviewer  has  obtained  information  or  explanations  from  the  respondent, 
hopefully,  some  in  the  form  of  the  respondent’s  own  terminology  or  “native  language.”  All 
different  kinds  of  questions  have  been  asked.  An  interview  differs  from  a  basic  conversation 
in  that  there  is  not  equal  turn-taking.  It  is  acknowledged  and  accepted  that  the  interviewer 
is  there  to  hear  the  respondent  talk.  The  interviewer  has  continued  to  express  interest  while 
professing  ignorance,  and  thus  the  importance  of  the  respondent’s  comments.  The  interviewer 
has  repeated  many  of  the  respondent’s  comments  and  restated  them  to  confirm  understand¬ 
ing.  The  interviewer  has  tried  to  incorporate  the  respondent’s  semantics  into  subsequent  probe 
questions.  The  interviewer  may  have  also  created  hypothetical  situations  to  test  their  under¬ 
standing.  Most  importantly,  the  interviewer  has  not  left  without  understanding  the  answers 
provided. 


Adapted  from  Spradley,  1979. 
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Closing  the  Interview 

•  Express  appreciation 

-  Indicate  how  useful  and  productive  the  session  has  been 

•  Confirm  next  steps 

•  Leave  professional  contact  information 

•  Snowball  sample 

•  Follow-up 

-  Send  notes  back  to  participant  for  comment 

-  Thank  you  note 
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The  interviewer  should  always  close  on  a  positive  note.  Let  the  respondent  know  how  useful  the 
interview  has  been  and  how  much  his  or  her  participation  is  appreciated. 

Now  is  also  the  time  to  confirm  any  next  steps  to  the  research  process.  Any  plans  for 
follow-up  contact  should  be  discussed.  It  may  be  most  constructive  to  leave  that  door  open, 
for  example,  with  “Would  you  mind  if  I  called  you  back  if  I  thought  of  other  questions?”  If 
appropriate,  let  the  respondent  know  when  the  data  will  be  published  or  available  for  review 
and  how  they  might  receive  a  copy  if  interested. 

It  is  also  often  important  to  leave  contact  information  with  respondents,  in  the  event  that 
they  think  of  more  information  they  want  to  share.  If  the  project  employs  snowball  sampling, 
and  the  respondent  has  suggested  someone  else  to  speak  with,  the  conclusion  of  the  interview 
is  the  appropriate  time  to  obtain  that  contact  information.  The  interviewer  might  also  want  to 
ask  whether  the  respondent  is  willing  to  open  the  door  with  a  phone  call  to  the  person. 

A  frequent  question  is  whether  notes  should  be  sent  back  to  a  respondent  to  review.  The 
circumstances  and  the  potential  for  bias  should  be  considered.  When  interviewing  someone 
for  his  or  her  professional  expertise,  it  might  be  important  to  have  the  notes  reviewed  for 
accuracy.  This  serves  a  couple  purposes.  It  provides  the  interviewer  the  opportunity  to  write 
questions  into  the  notes  where  additional  information  is  needed.  It  also  gives  the  respondent 
a  greater  sense  of  involvement  (and  hopefully  of  comfort)  in  the  research.  Additionally,  this 
review  allows  the  respondent  to  buy  into  (or  correct)  what  is  understood  from  the  interview, 
and  it  reduces  problems  later  that  might  be  caused  from  disagreement  over  the  findings.  It  also 
compels  the  interviewer  to  write  up  the  notes  quickly. 
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Alternatively,  when  an  interview  addressed  personal  experiences,  attitudes,  and  opinions, 
notes  are  less  frequently  provided  to  the  respondent  for  review. 

Lastly,  decide  whether  a  thank  you  note  is  appropriate,  given  the  researcher’s  relationship 
with  the  respondent  and  the  status  of  the  respondent.  If  notes  are  being  sent  hack,  the  thank 
you  can  take  the  form  of  the  first  paragraph  of  the  email  or  letter. 
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Overview 

•  Introduction  to  data  collection  methods 
'Applications  from  the  real  world 

•  Semi-structured  interviews 

—  Frame  the  research 
—  Sampling 

-  Designing  questions  and  probes 

-  Developing  the  protocol 

—  Preparing  for  the  interview 
—  Conducting  the  interview 

-  Capturing  the  data 

•  Focus  groups 

•  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 
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This  portion  of  the  course  reviews  methods  of  capturing  data  from  interviews. 
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Capturing  the  Data 

•  Sample  file 

•  Recording 

—  Whether  to  use  audio  recording 

-  Does  not  capture  everything 

-  Logisticai  issues 

•  Note-taking 
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There  should  be  several  sources  of  data  from  the  research  interviews. 

One  source  will  be  a  file  that  includes  information  on  the  sample  of  people  selected  for  the 
study.  These  files  also  often  contain  basic  information  about  the  sampling  criteria,  and  perhaps 
basic  demographic  information.  This  file  is  often  used  to  report  such  things  as  response  rates 
(of  the  people  selected,  how  many  completed  the  interview)  and  to  describe  the  characteristics 
of  the  population  and  respondents.  Tfiis  kind  of  data  is  very  important  in  bounding  the  asser¬ 
tions  that  can  be  made  with  the  data. 

Whether  or  not  to  record  an  interview  often  comes  up  as  a  question.  Some  things  to  think 
about  here  include  the  time,  cost,  and  effort  to  transcribe  the  tapes  and  the  way  tapes  must  be 
protected  (audio  tapes  are  considered  identifiable  data).  Researchers  also  need  to  be  aware  that 
tapes  and  the  resulting  transcripts  do  not  capture  all  the  data.  Notes  are  still  necessary  to  cap¬ 
ture  nonverbal,  background,  and  situational  data. 

Finally,  there  are  the  notes  that  were  taken  during  the  interview.  These  include  both  the 
interviewer’s  general  notes  and  the  more  detailed  notes  taken  by  the  note-taker. 
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Drafting  Interview  Notes 

•  Begin  interview  notes  with  date,  location, 
participants,  and  participants’  organization 

-  If  participant  name  cannot  be  included  in  notes,  use  a 
code  identifier  for  the  interviewee 

*  Interview  notes  should  have  an  overview  and 
background  introduction 

—  Background,  to  include  why  interviewing  this  individual 

-  Attitude  (e.g.,  enthusiasm,  resistance)  of  interviewee 

—  Additional  information  not  apparent  from  interview  content 

-  Location 

-  Distractions 
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Interview  notes  should  begin  with  descriptive  information:  the  date  and  location  of  the  inter¬ 
view;  the  participants,  including  the  interview  team;  and  information  about  the  participant’s 
organization.  For  some  projects,  conhdentiality  or  other  concerns  prevent  the  use  of  partici¬ 
pant  identihers,  such  as  name  or  organization.  In  these  cases,  code  identifiers  should  be  created 
and  included  in  the  notes. 

Interview  notes  should  also  include  information  on  why  the  respondent  was  included 
in  the  research;  for  example,  perhaps  the  individual  is  a  subject-matter  expert.  It  may  also  be 
important  to  note  any  issues  with  the  respondent  that  might  have  influenced  the  data  col¬ 
lected.  Was  the  respondent  particularly  resistant  to  the  interview  or  a  subset  of  questions? 
Finally,  be  sure  that  notes  include  any  additional  information  that  will  be  important  to  the 
analysis  that  will  not  be  included  in  the  actual  content  of  the  discussion.  Where  was  the  inter¬ 
view  conducted — in  a  private  office  or  a  public  space?  What  distractions,  if  any,  were  there 
during  the  interview? 
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Writing  Finai  interview  Notes 

*  Write  the  notes  with  polished  language  and  accurate 
spelling 

-  “You”  and  “are”  both  spelled  with  three  letters 

-  Complete  sentences 

*  Indicate  editorial  comments  differently  (e.g.,  in 
brackets) 

-  Interpretations,  explanation  of  comments 

—  Perceptions  of  attitude  when  saying  something 

*  Do  additional  work  to  confirm  factual  information  such 
as  names,  locations 

—  Google  to  confirm  spelling  of  prominent  names 

*  Indicate  when  notes  include  exact  quotes 

—  Avoid  quotes  that  are  not  meaningful,  or  place  them  in  useful  context 

*  Use  actual  names  (individual,  office,  location)  whenever 
possible;  avoid  relative  words  like  “they”  and  “here” 
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Final  interviewer  notes  should  be  written  with  polished  language,  with  words  and  terms  fully 
spelled  out.  Consider  that  notes  may  be  reviewed  sometime  after  the  interview  and  may  be 
used  by  a  researcher  who  was  not  in  attendance  for  the  actual  interview.  Notes  should  be 
understandable  if  reviewed  by  someone  years  after  the  research  is  complete. 

It  is  important  to  indicate  the  difference  between  actual  answers  gathered  from  the  respon¬ 
dent  and  information  added  by  either  the  interviewer  or  note-taker.  Therefore,  any  editorial 
comments  should  be  noted  in  way  that  makes  these  differences  clear  to  the  reader. 

If  appropriate,  the  note-taker  should  check  and  edit  all  factual  information.  Also  impor¬ 
tant  is  to  specify  exact  quotes,  but  only  those  that  are  meaningful.  Quotes  should  also  be  put  in 
context  of  the  discussion,  so  that  the  research  team  can  use  them  appropriately  in  the  analysis 
and  reporting. 

Finally,  whenever  possible,  notes  should  include  actual  names  and  avoid  vague  references 
such  as  “they”  and  “here.” 
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Exercise  3:  Interview  Notes 

•  Review  sample  notes 

*  Discuss  conventions 
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In  Exercise  3,  you  will  review  a  sample  of  interview  notes.  The  exercise  materials  and  the 
accompanying  discussion  identify  some  conventions  used  in  note-taking  and  highlight  exem¬ 
plars.  The  exercise  is  provided  on  pp.  121—122. 
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Overview 

•  Introduction  to  data  collection  methods 

•  Applications  from  the  real  world 

•  Semi-structured  interviews 

•  Focus  Groups 

-  What  are  focus  groups? 

-  When  and  why  to  conduct  focus  groups? 

-  Sampling  and  composition 

-  Recruiting  for  focus  groups 

-  Designing  questions  and  probes 

-  Conducting  the  focus  groups 

-  Capturing  the  data 

•  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 
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We  now  proceed  to  discuss  focus  groups.  This  focus  group  section  of  this  course  is  shorter 
than  the  preceding  interview  section.  That  is  due,  in  large  part,  to  the  core  methodological 
issues  that  are  appropriate  to  both  interviews  and  focus  groups,  such  as  question  construction 
and  remaining  neutral,  which  have  already  been  addressed  in  the  context  of  semi-structured 
interviews. 
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Definition  of  Focus  Groups 

•  Definitions  vary 

*  Defining  factors  include 

—  Whether  sessions  include  research 
—  Whether  there  is  a  moderator 
—  Number  of  participants 
—  Whether  participants  know  one  another 

-  Whether  participants  are  present 

—  Whether  there  is  dynamic  verbal  and  nonverbal 
discussion 

-  Type  of  venue 
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As  mentioned  previously,  focus  groups  are  dynamic  group  discussions  used  to  collect  informa¬ 
tion.  But  what  does  this  actually  mean,  and  how  does  this  compare  with  an  interview  when 
multiple  participants  are  present? 

One  useful  analogy  employs  the  notion  of  a  soccer  coach  and  team,  where  the  coach  rep¬ 
resents  the  interviewer,  the  players  are  the  respondents,  and  the  hall  represents  the  questions  to 
he  asked.  During  practice,  the  coach  may  kick  the  hall  to  one  player,  who  kicks  it  hack  to  the 
coach.  The  coach  may  then  kick  it  hack  to  the  same  player,  or  to  another,  who  also  then  returns 
it  to  the  coach.  In  this  instance,  the  coach  is  controlling  the  hall.  This  is  an  interview. 

A  focus  group  shares  more  characteristics  with  a  soccer  game.  During  a  game,  the  coach 
is  on  the  sideline,  and  the  hall  is  in  play  among  the  players.  The  players  move  the  hall  around 
among  themselves,  and  the  coach  encourages  the  action  from  the  sideline.  The  coach  may  call 
different  players  into  the  game,  or  pull  players  out,  hut  he  is  directing  and  not  playing.  This 
is  consistent  with  a  focus  group  in  that  the  coach  is  moderating  the  game,  and  the  players  are 
dynamically  interacting  among  themselves,  with  the  hall. 

Among  the  published  research,  definitions  of  what  is  actually  a  focus  group  vary  (see,  for 
example,  Morgan  and  Krueger,  1998;  Bernard,  2002;  Krueger  and  Casey,  2000),  hut  some 
of  the  defining  characteristics  include  that  the  interaction  must  include  research.  This  differs 
from  a  session  used  to  promote  a  product,  to  brainstorm,  or  used  when  the  leader  has  no  inten¬ 
tion  of  using  findings  but  is  trying  make  participants  believe  they  have  input. 

Focus  groups  include  a  moderator,  who  directs  the  conversation,  encourages  participa¬ 
tion,  and  makes  sure  that  data  are  collected  on  the  research  topic. 
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The  number  of  participants  is  also  an  important  consideration.  If  there  are  too  few,  then 
the  conversation  is  not  dynamic  and  generally  proceeds  more  like  an  interview.  If  there  are 
too  many,  then  participants  may  not  have  the  opportunity  to  fully  engage  with  one  another 
around  the  topic.  Though  there  are  no  precise  prescriptions,  focus  groups  generally  include  6 
to  11  participants. 

Some  researchers  contend  that  focus  group  participants  cannot  be  friends  or  acquain¬ 
tances  with  one  another.  In  reality,  there  are  situations  where  it  would  be  very  difficult  to 
ensure  that  participants  do  not  know  one  another.  For  instance,  if  research  is  conducted  within 
a  corporation  or  at  a  military  facility,  then  chances  are  at  least  some  of  the  participants  will 
know  one  another.  In  these  situations,  the  moderator  needs  to  take  care  to  mitigate  any  effects 
caused  by  participants  who  know  one  another,  and  the  researchers  need  to  be  aware  of  any 
acquaintanceship  in  order  to  take  it  into  account  in  their  analysis. 

Modern  communications  technologies  allow  multiple  people  to  interact  with  one  another 
from  a  distance.  Data  collection  firms  may  talk  about  “virtual”  focus  groups,  or  focus  groups 
that  are  conducted  by  conference  call  or  video  conference.  However,  focus  groups  require 
dynamic  interaction,  both  verbal  and  nonverbal;  Internet  or  telephone  sessions  do  not  include 
nonverbal  participation  and  thus  do  not  satisfy  a  basic  definitional  requirement  for  focus 
groups.  While  such  sessions  may  be  useful,  they  are  not  focus  groups.^' 

Finally,  the  type  of  venue  for  the  focus  group  has  to  be  considered,  and  some  researchers 
claim  that  sessions  can  only  be  considered  focus  groups  when  convened  in  a  focus  group  facil¬ 
ity.  These  specially  designed  focus  group  suites,  which  have  one-way  mirrors  and  high-quality 
recording  equipment,  are  the  gold  standard,  but  they  are  not  a  requirement,  and  many  focus 
groups  are  conducted  on-site  with  the  resources  available. 


The  requirement  for  in-person,  dynamic  interaction  is  often  debated,  and  other  researchers  may  not  agree  with  this 
requirement. 
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Purpose  of  Focus  Groups 

•  Use  in  different  phases  of  project 

•  Test  survey  questions  or  develop  semantics 

•  Explain  survey  results 

•  Provide  insights  into  seemingly  conflicting 
opinions 

•  Explore  WHY  people  feel  that  way 

•  Never  provide  statistical  “counts” 
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Focus  groups  can  serve  multiple  purposes.  Focus  groups  are  sometimes  used  as  part  of  ques¬ 
tionnaire  development,  for  testing  or  developing  questions,  or  for  examining  the  meaning  of 
words  or  phrases. 

Focus  groups  can  also  he  used  to  help  explain  results  found  through  other  data  collection 
methods,  such  as  surveys.  Focus  groups  are  especially  helpful  in  explaining  findings  that  appear 
counterintuitive  or  conflicting.  Groups  can  also  be  asked  to  explore  how  things  happen,  or  why 
people  feel  a  certain  way.  One  can  also  ask  participants  to  discuss  differences  of  opinions. 

Focus  group  data  can  be  very  rich,  but  an  important  thing  to  understand  is  that  focus 
groups  can  never  provide  statistical  counts.  Researchers  can  never  say  that  67  percent  of  the 
people  who  participated  in  our  focus  groups  felt  a  certain  way.  The  researchers  do  not  know 
how  everyone  in  the  group  felt.  Researchers  know  what  was  said  by  one  or  more  respondents 
during  the  group,  but  they  are  unable  to  know  the  attitudes  of  all  the  participants.  In  addi¬ 
tion,  due  to  the  dynamic  aspects  of  the  discussion,  the  researchers  can  never  know  the  extent 
to  which  one  respondent  might  influence  the  answers  of  another. 
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Group  Sampling  and  Composition 

•  Consider  your  research  question:  Who  should  be 
included  in  the  study? 

*  What  characteristics  do  you  want  to  analyze  by? 

-  Common  characteristics  of  groups 
—  Will  be  basis  for  analysis 
—  Promotes  conversation 

—  Differing  characteristics  within  groups  should  also  be 
considered 

—  Demographic  characteristics 

-  Power  structure  placement 

-  Level  of  acquaintance 

—  Physicai  and  cognitive  abiiity 
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Stewart  and  Shamdasani  (1990)  provide  an  in-depth  discussion  of  considerations  that  should 
he  taken  into  account  when  determining  the  sampling  and  composition  of  focus  groups.  Here 
are  some  considerations. 

In  deciding  whom  to  include  in  the  focus  groups,  researchers  need  to  look  hack  to  their 
research  questions.  They  need  to  consider  who  can  best  address  the  research  questions.  They 
also  need  to  consider  whether  a  group  discussion  makes  sense  in  terms  of  the  topics  and  the 
individuals.  Are  the  topics  too  sensitive,  and  will  this  population  he  comfortable  discussing 
these  issues  in  a  group?  Generally  speaking,  very  sensitive  topics,  such  as  sexual  behavior,  are 
not  good  topics  to  explore  in  focus  groups. 

Can  the  researchers  convene  a  group  of  participants?  Researchers  might  be  able  to  get  a 
group  of  enlisted  soldiers  in  the  same  room  at  the  same  time,  but  could  they  really  gather  a 
group  of  generals? 

When  deciding  on  the  group  composition,  researchers  need  to  determine  the  characteris¬ 
tics  by  which  they  want  to  analyze.  If  the  study  will  compare  the  opinion  of  men  and  women 
regarding  a  policy,  then  separate  groups  of  men  and  women  are  needed.  Likewise,  if  analysis 
will  consider  the  different  perspectives  of  individuals  based  on  other  demographic  characteris¬ 
tics,  such  as  race/ethnicity,  occupation,  or  age,  then  the  focus  groups  will  need  to  sample  based 
on  those  characteristics.  Researchers  cannot,  for  example,  mix  men  and  women  in  the  same 
group  and  subsequently  compare  them  in  the  analysis.  Nor  can  researchers  compare  the  com¬ 
ments  of  particular  individuals  (e.g.,  women)  from  one  group  with  the  same  type  of  individu¬ 
als  from  another  group  in  order  to  assert  what  that  type  of  individual  said  throughout  focus 
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groups.  For  instance,  in  a  recent  study,  a  series  of  focus  groups  was  conducted  with  patients 
that  had  the  same  disease.  Patients  with  early-  and  regular-onset  of  the  disease  participated  in 
separate  groups.  This  was  done  because  the  researchers  thought  that  there  might  he  substan¬ 
tial  differences  in  quality  of  life  between  these  two  different  types  of  patients  and  wanted  to 
compare  them. 

Within  a  group,  researchers  may  need  to  consider  whether  a  mix  of  demographic  char¬ 
acteristics,  such  as  age,  race,  or  gender,  is  desirable.  It  is  important  to  consider  what  character¬ 
istics  might  affect  the  discussion,  and  how.  Consideration  should  be  given  to  what  promotes 
conversation  within  a  group.  There  may  be  issues  related  to  the  power  structure  of  the  group. 
For  instance,  researchers  often  segment  military  groups  by  junior  and  senior  ranks  and  by 
enlisted  personnel  and  officers.  As  one  can  imagine,  a  junior  enlisted  soldier  may  feel  less  com¬ 
fortable  speaking  out  against  military  policies  with  his  commander  in  the  room. 

Researchers  should  also  consider  the  level  of  acquaintance  among  participants.  If  people 
know  each  other,  then  there  is  a  risk  of  side  conversations,  unspoken  language,  and  shared 
background  to  contend  with.  Ideally,  the  participants  would  not  know  each  other,  but  this  is 
not  always  possible. 

Finally,  issues  such  as  physical  and  cognitive  abilities  should  be  considered.  For  a  recent 
health  study,  focus  groups  with  caregivers  were  conducted  because  a  subset  of  patients  had 
decreased  cognitive  abilities.  The  researchers  thought  it  was  important  to  address  the  expe¬ 
riences  of  these  patients  in  the  study,  but  the  patients  were  themselves  unable  to  participate 
in  focus  groups.  In  this  instance,  caregivers  provided  information  on  behalf  of  this  group  of 
patients. 
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Number  of  Focus  Groups  Necessary? 

*  No  good  answer  to  this  question 

*  Depends  on  purpose  of  focus  groups 

*  Look  for  convergence 

*  Be  wary  of  drawing  conclusions  -  suggestive, 
insightful,  but  never  generalized 

*  Sometimes  (especially  in  DoD)  it  is  easier  to 
schedule  more  than  you  might  need  than  to 
return 
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Perhaps  the  single  most  frequently  asked  question  about  focus  groups  is,  How  many  focus 
groups  are  necessary?  Unfortunately,  there  is  no  simple  answer  to  this  question. 

The  answer  lies  partly  in  the  purpose  of  the  groups — for  instance,  are  they  being  con¬ 
ducted  to  develop  items  for  a  survey  or  for  primary  data  collection?  How  many  subgroups  need 
to  be  included  in  order  to  address  the  key  common  characteristics? 

Once  these  factors  are  decided,  logistics  and  budget  are  also  inputs  to  this  decision. 

Ideally,  researchers  will  conduct  focus  groups  and  assess  whether  they  have  obtained  con¬ 
vergence  across  the  groups,  to  determine  whether  they  have  conducted  enough  focus  groups. 
In  other  words,  if  the  researchers  begin  to  hear  similar  themes,  with  no  new  issues  being  raised, 
then  they  have  probably  conducted  a  sufEcient  number  of  focus  groups. 

Please  note  that  focus  group  data  can  be  very  insightful,  but  they  only  indicate  the  views 
of  the  entire  group.  Focus  group  data  can  be  quantified  across  the  types  of  groups  that  were 
conducted,  based  on  the  common  characteristics.  However,  researchers  can  never  generalize 
focus  group  data  to  the  broader  population. 

In  some  circumstances,  such  as  when  preauthorization  to  schedule  groups  is  required  or  it 
takes  a  long  time  to  recruit  participants,  it  is  best  to  plan  for  more,  rather  than  fewer,  groups. 
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Formulating  Focus  Group  Protocols 

*  Develop  a  precise  protocol 

-  Especially  important  with  numerous  moderators 

*  Prioritize  research  questions 

*  Allocate  group  discussion  time  to  each  topic  area 

*  Consider  structural  or  descriptive  questions 

—  Avoid  “yes-no”  and  simple  answers 

*  Emphasize  proving  knowledge  rather  than 
acknowledging  ignorance 

*  Avoid  threatening  or  embarrassing  questions 
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As  with  interviews,  focus  group  protocols  are  extremely  important,  to  ensure  consistency  across 
multiple  moderators,  in  prioritizing  research  questions,  and  in  allocating  group  discussion  time 
across  topic  area. 

Careful  consideration  needs  to  he  given  to  the  types  of  questions  asked  during  focus 
groups.  Structural  questions  will  provide  lists  of  things,  which  may  he  useful  if  the  researchers 
are  designing  a  survey  and  need  to  know  all  the  different  possible  answers.  Descriptive  ques¬ 
tions  will  provide  a  greater  understanding  than  researchers  could  likely  derive  from  surveys, 
so  these  types  of  questions  are  appropriate  if  the  focus  groups  are  being  used  to  add  depth  to 
quantitative  analysis. 

Simple  and  yes-no  questions  are  likely  to  be  extremely  boring  and  can  make  the  mod¬ 
erator  look  silly  if  they  are  repetitive.  There  are  additional  problems  with  yes-no  questions,  as 
many  participants  may  either  be  reluctant  to  differ  from  the  group  (be  the  only  yes)  or  admit 
lack  of  knowledge.  For  example,  it  is  better  to  ask  participants  to  create  a  list  of  items  than  to 
ask  whether  they  are  familiar  with  each  item  on  a  prepared  list.  The  latter  approach  is  likely  to 
produce  false  positive  acknowledgement  of  items  on  the  list. 

Finally,  focus  group  protocols  should  avoid  questions  that  might  make  participants  overly 
uncomfortable.  Remember  that  the  purpose  of  the  focus  group  is  to  engage  all  participants  in 
the  topic  under  discussion.  Questions  should  not  be  threatening  or  embarrassing,  and  should 
not  “test”  the  participants.  Questions  should  emphasize  the  participants’  knowledge  about  a 
topic  rather  than  make  them  feel  unknowledgeable. 
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Typology  of  Focus  Group  Questions 

*  Background  or  “icebreaker” 

*  Main  research  questions 

*  Factual  questions 

*  Anonymous  questions 

*  Kitchen  sink  question 

*  Big  picture  question 
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There  are  some  questions  that  are  unique  to  focus  groupsT^  The  background  or  icebreaker  ques¬ 
tion  starts  off  the  discussion  and  is  meant  to  get  all  participants  comfortable  talking.  Often 
this  question  is  also  used  to  obtain  important  information  about  the  characteristics  of  the  par¬ 
ticipants,  such  as  their  occupation. 

Next  are  the  main  research  questions,  what  the  moderator  is  there  to  collect. 

If  the  focus  group  conversation  mentions  something  the  moderator  is  not  aware  of,  such 
as  a  program  or  policy,  the  moderator  might  ask  a  factual  question  to  understand  the  discus¬ 
sion  better. 

Anonymous  questions  may  be  used  when  a  subject  matter  is  sensitive  or  controversial. 
Here  the  moderator  asks  participants  to  write  their  answers  on  index  cards  and  collects  them, 
either  for  later  analysis  or  to  fuel  the  conversation  without  attributed  responses  to  individual 
participants. 

The  kitchen  sink  question  generally  comes  toward  the  end.  It  permits  respondents  to  talk 
about  issues  of  importance  to  them  that  have  not  come  during  the  discussion.  It  might  be 
posed  as  such,  “Are  there  any  other  issues  that  we  haven’t  touched  upon  that  you  would  like  to 
discuss?”  The  promise  of  a  later  opportunity  to  discuss  additional  topics  is  also  helpful  to  the 
moderator  in  controlling  the  group:  “That’s  very  interesting,  let’s  talk  about  that  at  the  end  of 
our  discussion.” 


Adapted  from  Stewart  and  Shamdasani,  1990. 
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The  big  picture  question  usually  comes  at  or  near  the  end  of  the  interview  and  can  be 
helpful  in  uncovering  themes  and  issues  not  previously  considered.  For  example,  “This  research 
effort  about  your  university  library  has  been  funded  by  the  Board  of  Trustees  of  this  university. 
If  you  had  an  opportunity  to  take  your  own  message  on  this  topic  to  the  Board  of  Trustees, 
what  would  you  say?” 
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Focus  Group  Probes 

•  Most  interview  probes  are  effective  in  groups 

•  Probe  during  focus  groups  to 

-  Steer  conversation  to  topics 

-  Control  participants 

-  Encourage  dynamic  conversation 

*  Does  anyone  have  an  example  of  that? 

*  Has  anyone  else  experienced  this? 

*  You  look  puzzled  by  her  comment.  Why? 

*  Does  anyone  have  a  different  experience/perspective? 
{Rather  than  Does  anyone  disagree?) 

•  Be  aware  of  the  sensitivity  of  probing  when  there  are 
other  people  present 
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Many  of  the  probes  used  for  interviewing  are  also  appropriate  to  use  during  a  focus  group. 
However,  there  are  also  probes  that  are  specific  to  focus  groups;  such  probes  can  be  used  to 
keep  the  conversation  moving  on  topic,  take  control  away  from  a  dominant  person,  or  draw 
out  some  of  the  more  quiet  participants. 

One  of  the  goals  of  the  focus  group  is  to  include  all  participants  in  the  discussion,  not  just 
the  most  talkative  or  the  most  opinionated.  Probes  such  as  “I  hear  what  you  are  saying,  Ken, 
now.  I’d  like  to  hear  from  some  other  folks  about  that  issue”  or  “What  do  others  think  about 
this  issue?  Mary,  how  about  you?”  can  provide  an  opening  for  others  to  speak. 

Another  caveat  is  important  in  this  context,  because  participation  is  voluntary  and  no  one 
should  be  forced  to  speak  when  they  are  not  comfortable.  A  moderator,  when  trying  to  draw 
out  a  quiet  person,  should  not  make  reluctant  participants  uncomfortable.  Thus,  it  is  impor¬ 
tant  that  the  moderator  look  for  nonverbal  language  to  help  determine  when  it  is  appropriate 
to  pursue  input. 


90  Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups 


Exercise  4 

•  Review  focus  group  protocol 

*  Identify  problems  and  offer  improvements 
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In  Exercise  4,  you  will  review  a  sample  focus  group  protocol,  identify  some  problems  with 
the  protocol,  and  offer  potential  improvements  for  the  protocol.  The  exercise  is  provided  on 
pp.  123-126. 
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Conducting  the  Group 


•  Possible  locations 

•  Research  team 

-  Role  of  moderator 

-  Role  of  note-taker 

•  Logistical  support 

•  Issue  of  observers 
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There  are  some  additional  considerations  that  researchers  should  take  into  account  when  plan¬ 
ning  focus  groups. 

The  ideal  space  to  conduct  a  focus  group  is  in  a  facility  specifically  designed  for  these 
types  of  discussions.  In  reality,  moderators  often  find  themselves  conducting  these  groups  in 
a  wide  variety  of  settings.  Many  times,  a  conference  room  is  more  than  adequate.  As  with 
interviews,  what  is  really  important  is  that  the  space  be  quiet  and  private.  It  is  also  helpful  to 
conduct  the  focus  group  in  a  room  where  the  chairs  can  be  moved  around  a  table  or  to  form 
a  circle,  as  opposed  to  an  auditorium  or  training  facility,  where  the  chairs  are  often  secured  to 
the  floor  and  facing  forward. 

Two  people  that  must  be  in  the  room  are  the  moderator  and  the  note-taker.  The  modera¬ 
tor  should  be  able  to  focus  on  the  discussion;  his  or  her  job  is  to  get  the  data.  He  or  she  should 
be  able  to  manage  the  interaction  in  the  room,  steering  participants  as  needed,  and  determine 
when  topics  have  been  sufficiently  discussed  and  it  is  time  to  advance  to  another  topic.  Ideally, 
the  moderator  should  not  have  to  focus  on  logistics,  note-taking,  or  anything  else  once  the 
group  begins. 

The  note-taker  should  be  focused  on  capturing  the  details  of  the  conversation  and  non¬ 
verbal  behavior.  He  or  she  should  have  a  sense  of  when  it  is  appropriate  to  interject.  For  this 
reason,  it  is  helpful  have  a  third  person  to  provide  logistical  support,  check  participants  in,  deal 
with  administrative  issues,  and  handle  latecomers,  but  often  research  efforts  are  unable  to  sup¬ 
port  a  third  person. 
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The  question  of  whether  to  allow  observers  often  arises.  The  goal  of  the  focus  group  is  to 
have  a  free-flowing  discussion.  Too  many  nonparticipating  people  in  a  room  can  inhibit  con¬ 
versation.  In  general,  there  should  be  a  minimal  number  of  individuals  in  the  room  other  than 
participants.  Consideration  needs  also  to  be  given  to  the  position  of  the  observer.  For  example, 
a  corporate  executive  observing  a  focus  group  of  workers  would  likely  compromise  the  value 
of  the  discussion.  However,  there  are  valid  reasons  to  involve  an  observer,  such  as  for  training 
purposes  within  the  research  effort.^^ 


One  benefit  of  a  professional  focus  group  facility  is  that  observers  are  hidden  from  the  group  participants. 
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Common  Problems  with  Participants 

•  Experts  -  legitimate  or  self-appointed 

•  Rank  or  hierarchy 

•  Friends 

•  Hostile  group  members 

•  Unclear  whether  they  meet  the  sampling 
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During  the  course  of  the  focus  group,  a  moderator  is  often  called  upon  to  mitigate  the  influ¬ 
ence  of  certain  individuals  who  might  have  undue  influence  over  the  group  interaction.  The 
importance  of  a  skilled,  experienced  moderator  cannot  he  overestimated  in  these  instances.^"^ 

For  example,  a  participant  may  he  a  real  or  self-proclaimed  expert  on  the  topic  under  dis¬ 
cussion.  If  other  participants  feel  intimidated  or  unknowledgeahle  relative  to  such  an  individ¬ 
ual,  dynamic  conversation  may  cease.  The  moderator  needs  to  allow  the  expert  to  he  heard,  hut 
know  when  and  how  to  “turn  them  ofF’  and  allow  or  encourage  others  to  speak.  A  similar  need 
may  arise  if  individuals  perceive  one  another  to  he  of  different  social  rank  or  different  places 
within  a  social  or  corporate  hierarchy,  although  ideally  the  researchers  would  have  anticipated 
these  relationships  and  designed  the  groups  to  avoid  them. 

A  moderator  may  also  need  to  separate  friends  prior  to  the  start  of  the  group  in  order  to 
avoid  side  conversations  or  so  that  other  participants  do  not  feel  excluded  or  outnumbered. 

Some  participants  have  no  real  desire  to  participate  productively  in  the  group  interaction. 
They  may  appear  hostile  or  threatening  and  can  damage  a  group’s  interaction.  Sometimes  a 
skilled  moderator  can  convert  this  person  with  humor  or  understanding;  however,  if  this  is  not 
possible,  the  moderator  may  need  to  ask  the  person  to  leave.  Often  a  moderator  will  use  a  five- 
minute  break  to  privately  dismiss  a  problem  individual  from  the  group  (Morgan  and  Krueger, 
1998).  To  defuse  the  dismissal,  a  strategy  might  be  to  ask  the  individual  to  contact  the  modera¬ 
tor  at  a  future  time  when  the  moderator  can  “fully  consider  their  opinions.” 
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Adapted  from  Stewart  and  Shamdasani  (1990)  and  Morgan  and  Krueger  (1998). 
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On  occasion,  someone  may  attend  who  appears  not  to  meet  the  sampling  criteria  for  the 
group.  If  the  researchers  have  conducted  their  own  sampling,  and  if  they  check  the  participants 
against  the  list  of  expected  participants,  this  issue  will  he  immediately  addressed.  However, 
there  are  many  instances,  especially  when  conducting  focus  groups  in  defense  organizations, 
when  a  local  individual,  such  as  someone  from  the  installation,  has  provided  personnel  accord¬ 
ing  to  the  researchers’  guidance — e.g.,  Hispanic  female  personnel,  age  20  to  30 — hut  perhaps 
a  person  does  not  appear  to  he  the  correct  ethnicity  or  age.  In  these  awkward  situations,  the 
moderator  can  announce,  without  focusing  on  the  suspect  participant,  the  presumed  content 
of  the  group:  “This  is  the  11:00  group,  which  consists  of  Hispanic  women  between  the  ages  of 
20  and  30.”  Given  such  an  announcement,  any  participant  that  does  not  meet  the  sampling 
will  usually  identify  herself  so  that  the  moderator  can  excuse  them.  In  some  instances,  indi¬ 
viduals  who  were  unahle  to  attend  their  scheduled  time  have  been  known  to  appear  at  a  dif¬ 
ferent  time,  presuming  that  all  focus  groups  are  the  same.  This  approach  would  permit  those 
potentially  inappropriate  individuals  to  identify  themselves. 
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Capturing  the  Results 


•  Recording 

•  Transcript 

•  Moderator’s  notes  and  written  impressions 

•  Note-taker’s  formal  notes 


-  Participants  noted  by  number 

-  Nonverbai  aspect  to  notes 

•  Consensus 

•  Body  language 

*  Any  additional  data 

-  Recruitment  data 

-  Pre-focus  group  background  questionnaire 


RAND 


Some  researchers  believe  that  all  focus  group  sessions  should  always  be  audio-  or  video-recorded, 
while  others  are  comfortable  working  from  notes.  Regardless  of  the  approach,  it  is  important  to 
remember  that  no  video  or  audio  reproduction  captures  every  element  of  the  session.  Nonver¬ 
bal  behaviors  cannot  be  captured  by  voice  recording,  and  they  may  not  be  captured  in  a  video 
recording  (for  example,  if  the  subject  is  off-camera).  Also  consider  that  recordings  must  be 
transcribed,  and  thus  researchers  need  to  have  adequate  resources  available  for  transcription. 

During  the  focus  group  session,  the  moderator  will  take  general  notes.  Sometimes  a  mod¬ 
erator  takes  responsibility  for  noting  exact  quotes  during  the  discussion.  Often  the  moderator 
is  responsible  for  drafting  an  overall  assessment  of  the  group  interactions  immediately  fol¬ 
lowing  a  focus  group  session.  The  moderator’s  notes  and  impressions  should  be  cleaned  and 
included  in  a  data  file  for  analysis. 

As  with  interviews,  focus  group  notes  should  begin  with  descriptive  information.  In  addi¬ 
tion  to  the  date  and  location  of  the  focus  group  and  critical  background  information,  the 
note-taker  should  assign  a  number  to  each  participant  so  the  conversation  can  be  tracked  at 
the  participant  level.  Note-takers  should  record  information  on  nonverbal  behaviors  and  body 
language  of  participants.  Instances  where  consensus  was  reached  among  participants  or  where 
disagreement  existed  among  the  group  should  also  be  noted. 

In  addition  to  transcripts  and  notes,  such  data  as  recruitment  information  should  be 
assembled  and  included  in  the  analysis.  Sometimes  researchers  ask  participants  to  complete 
a  pre-focus  group  questionnaire  that  collects  demographic  or  background  information  from 
participants  prior  to  the  group.  This  data  may  also  be  coded  and  used  in  the  analysis. 
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Exercise  5 

*  Review  focus  group  transcript  and  notes 

*  Discuss  exemplars 

*  Identify  problems 


RAND 


In  Exercise  5,  you  will  review  excerpts  from  a  transcript  and  detailed  notes  created  from  a 
sample  focus  group,  discuss  exemplars,  and  identify  some  problems  with  the  notes  provided. 
The  exercise  is  provided  on  pp.  127-134. 
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Overview 

•  Introduction  to  data  collection  methods 
'Applications  from  the  real  world 
•Semi-structured  interviews 

•  Focus  groups 

•  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 
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This  section  discusses  how  researchers  can  organize  and  manage  data  and  conduct  basic  analy¬ 
sis  of  qualitative  research  data. 
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Data  Organization 

*  Interview  or  focus  group  list  with  file  names 

*  Final  session  notes 

-  Includes  notes  as  well  as  background  and  impressions 

*  Data  status 

-  Date  occurred 

—  Whether  transcription  is  complete 
—  Who  is  reviewing  transcription  -  date  completed 

-  Who  is  primary  coder  -  date  completed 

—  Who/whether  there  is  secondary  coder  -  date  completed 


RAND 


Interviews  and  focus  groups  result  in  a  variety  of  electronic  files.  There  should  be  a  list  of  the 
sessions  conducted  with  the  file  names  for  each  session.  There  will  also  be  the  final  notes  from 
each  of  the  sessions.  Each  of  these  files  will  include  the  session  notes  as  well  as  the  background 
and  impressions  information.  Finally,  the  research  project  should  have  a  file  that  includes  infor¬ 
mation  about  the  status  of  the  research  data.  For  a  complex  project  involving  transcription, 
this  file  might,  for  example,  be  a  spreadsheet  that  includes  columns  recording  the  date  the  ses¬ 
sion  occurred,  whether  the  transcription  is  complete,  who  is  reviewing  the  transcription  and 
whether  the  review  is  complete,  the  name  of  the  primary  and  secondary  coders  and  whether 
the  coding  has  been  completed,  etc.  For  a  more  simple  effort,  this  file  might  include  informa¬ 
tion  regarding  the  interviewer  and  note-taker  and  the  status  of  the  notes. 
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File  Management 

•  Use  a  file-naming  scheme  that  is  useful  and  that 
indicates  most  relevant  information 

•  75  11  OSS  H 

-  Unique  identifier 

-  Unit  and  iocation 

-  Paygrade  and  spouse/member 

-  Race 

•  9204SVO5C2M 

-  Unique  identifier 

-  Location 

-  Unit  or  voiunteer 

-  Paygrade 

-  Education 

-  Work  status 
RAND“  Interviewer 


Often,  data-safeguarding  rules  preclude  interviews  from  being  labeled  using  the  respondent’s 
name.  In  these  cases  and  in  the  case  of  focus  groups,  it  is  useful  to  use  a  file-naming  scheme 
that  indicates  the  most  relevant  information  about  each  session.  This  slide  shows  two  examples 
from  prior  research  projects.  Both  examples  begin  with  a  unique  identifier,  which  is  always  best 
practice  given  the  possibility  (however  unlikely)  that  the  characteristics  reflected  in  the  name 
would  otherwise  be  alike.  In  the  first  example,  the  other  characteristics  reflected  in  the  naming 
convention  include  the  unit  and  location  of  the  respondent,  his  or  her  pay  grade,  whether  the 
respondent  was  a  service  member  or  a  spouse,  and  the  respondent’s  race.  There  exists  a  master 
sheet  to  indicate,  in  this  example,  the  meaning  of  location  1  and  unit  1,  that  OS5  means  the 
spouse  of  a  service  member  with  a  pay  grade  of  0-5,  and  that  H  indicates  Hispanic. 

The  second  example  involves  a  more  detailed  master  sheet  for  this  study  of  military 
spouses.  In  this  example,  the  naming  convention  begins  with  the  unique  identifier  (9204), 
which  is  followed  by  symbols  that  indicate  the  location,  that  the  respondent  self-presented  and 
volunteered  to  participate  in  the  study  (rather  than  volunteering  after  being  invited),  that  she 
was  the  spouse  of  an  0-5,  that  she  had  a  college  degree,  that  she  was  employed,  and  which 
researcher  conducted  the  interview. 

Such  naming  conventions  provide  basic  information  about  the  respondent.  This  is  espe¬ 
cially  helpful  when  extracting  statements  from  different  interviews  in  a  study  that  contains 
hundreds  of  interviews.  For  example,  if  the  researcher  has  quotes  or  excerpts  from  six  different 
interviews  and  knows  the  file  name  (usually  part  of  the  software  output),  the  researcher  has 
some  basic  information  about  each  interview. 
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Basic  Qualitative  Data  Analysis 

•  Deductive  analysis  confirms 

-  Gather  answers  to  same  questions 

-  Confirm  or  refute  hypotheses 

*  Inductive  analysis  explores 

-  Use  word  find  to  see  common  topics 

-  Identify  and  code  new  themes 
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When  conducting  qualitative  data  analysis,  there  are  two  basic  approaches  to  analyzing  data, 
and  many  projects  will  include  both  of  these  approaches. 

Deductive  analysis  conhrms  information  for  the  researcher.  Many  projects  use  deduc¬ 
tive  analysis  to  gather  together  answers  to  a  particular  question.  This  enables  the  researcher 
to  look  at  what  all  respondents  said  to  the  same  question.  Another  use  of  deductive  analysis 
is  to  confirm  or  refute  research  hypotheses,  or  interactions  within  the  data  that  the  researcher 
presumed. 

Inductive  analysis  explores  the  data,  to  assess  what  unexpected  relationships  or  issues 
emerge  from  the  data.  This  analysis  might  include  a  word  find  to  determine  the  most  common 
topics  mentioned  in  each  interview.  It  might  also  include  a  line-by-line  reading  to  identify 
themes  in  the  interviews  that  the  researcher  had  not  anticipated. 
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Text  Analysis  Process 

•  Select  text  to  examine 

•  Identify  themes 

•  Build  codebook 

•  Tag  text  for  themes 

•  Search  for  subthemes 

•  Search  for  patterns  among  themes 

-  Examine  distribution  of  themes  across  types  of  people/ 
cases 

-  Examine  relationships  among  themes  (potential 
hypotheses) 
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When  analyzing  text,  the  first  step  is  to  select  the  text  to  he  examined  and  to  identify  themes 
that  will  he  sought  in  the  data.^^  These  themes  might  he  identified  deductively,  such  as  in  the 
case  where  themes  are  associated  with  each  question  asked,  or  they  might  also  he  identified 
inductively  as  the  analysis  proceeds.  For  example,  in  one  recent  research  effort  addressing  the 
deployment  experiences  of  military  families,  the  researchers  found  that  the  effect  of  the  deploy¬ 
ment  on  children  emerged  in  many  of  the  interviews,  although  there  was  not  a  question  specifi¬ 
cally  asking  about  the  effect  on  children. 

The  terminology  for  identifying  codes  in  the  text  is  coding  or  tagging.  In  many  soft¬ 
ware  applications,  this  means  literally  selecting  text  and  associating  it  with  a  particular  theme 
or  code.  In  a  simpler  approach,  researchers  might  only  mark  text  with  colored  pencils  with 
colors  representing  different  themes  or  codes.  For  the  purpose  of  this  discussion,  the  method 
of  coding  is  not  important,  although  the  use  of  a  software  package  designed  to  code  text  will 
facilitate  some  of  the  data  manipulation  discussed  in  later  slides. 

After  the  researcher  identifies  basic  themes  in  the  data,  the  next  step  is  to  search  for  sub¬ 
themes.  For  example,  if  one  theme  is  the  positive  aspects  of  deployment  for  military  families, 
subthemes  might  include  the  different  kinds  of  positive  aspects. 

Once  data  are  tagged  or  coded  by  theme,  the  researcher  is  able  to  search  for  patterns 
among  the  themes.  This  might  mean  understanding  what  kinds  of  people  or  situations  led  to 
certain  themes.  For  example,  perhaps  only  people  in  one  community  mentioned  one  theme,  or 


Discussions  of  text  analysis  process  are  adapted  from  Ryan  and  Bernard,  2000;  and  Bernard  and  Ryan,  2009. 


102  Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups 


only  young  people  made  a  certain  assertion.  The  researcher  is  also  able  to  consider  relationships 
between  the  themes.  For  example,  perhaps  people  who  held  a  certain  opinion  were  most  likely 
to  agree  about  another  issue. 
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Basics  of  a  Coding  Tree 

1  Attribute  Codes 

2  Substantive  Codes 
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Coding  trees  are  the  list  of  themes,  or  codes,  that  will  he  applied  to  research  data.  Trees  are 
often  divided  into  two  different  types  of  codes.  Attribute  codes  reflect  the  respondent  or  focus 
group  participants  and  often  convey  demographic  information  about  that  individual  or  group. 
Substantive  codes  reflect  the  content  of  the  session. 
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Basics  of  a  Coding  Tree:  Attribute  Codes 


1  Attribute  Codes 

1.1  Gender 

\  1.1.1  Female 

\  1.1.2  Male 

\  1 .2  Parental  status 

\  1 .2.1  Has  children 

\  1 .2.2  No  children 

\  1.3  Education  level 

2  Substantive  Codes 

\  1.3.1  <  12  years 

\  1 .3.2  High  school 

\  1 .3.3  Some  college,  no  degree 

\  1 .3.4  Assoc  degree 

\  1 .3.5  Bach  degree 

H  .3.6  Grad  degree 

RAND 

In  this  example,  a  mock  study  including  interviews  with  spouses  of  deployed  reservists,  the  attri¬ 
bute  codes  shown  reflect  a  subset  of  the  different  demographic  characteristics  of  the  spouses.^'’ 
In  the  application  of  these  codes,  each  respondent  was  assigned  an  attribute  code  from 
each  of  these  three  lists.  For  example,  one  respondent  might  be  a  female  spouse  with  children 
who  has  taken  some  college  courses,  but  has  not  received  a  degree. 


The  example  shown  in  this  document  does  not  use  actual  data,  but  does  borrow  concepts,  including  coded  themes,  from 
an  actual  study,  Castaneda  et  ah,  2008. 
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Basics  of  a  Coding  Tree:  Substantive  Codes 

1  Attribute  Codes 

2.1  Definition  of  readiness 

2.2  How  ready  was  the  family? 

2.3  Positive  aspects  of 
deployment 

2.4  Negative  aspects  of 
deployment 

2.5  Resources  used  during 
deployment 
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2  Substantive  Codes 


The  second  part  of  the  coding  tree  consists  of  substantive  codes.  These  pertain  to  the  actual 
content  of  the  interview.  This  slide  indicates  substantive  codes  from  that  same  mock  project.  At 
the  two-digit  level  of  the  coding  tree,  the  themes  included  how  the  respondents  defined  readi¬ 
ness  for  their  family,  their  discussion  of  how  ready  the  family  was,  the  positive  and  negative 
aspects  of  deployment,  and  the  resources  their  family  used  during  the  deployment. 

Unlike  the  attributed  codes  that  are  applied  to  the  entire  session  of  notes,  substantive 
codes  are  applied  only  to  that  portion  of  the  session  pertaining  to  the  theme.  The  reason 
for  this  can  be  explained  by  example  queries.  Consider,  for  example,  the  difference  between 
saying,  “Give  me  all  the  statements  made  by  respondents  who  were  parents,”  which  would  be 
the  entire  set  of  notes  for  each  of  those  people,  and  saying  “Give  me  all  the  notes  discussing  the 
positive  aspects  of  deployment.”  The  latter  would  include  only  the  relevant  part  of  the  notes 
from  each  interview. 
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Basics  of  a  Coding  Tree:  Substantive  Codes 

1  Attribute  Codes 

2  Substantive  Codes 

2.3.1  Financial 

2.3.2  Patriotism,  pride, 
civic  responsibility 

2.3.3  Spouse  independence, 
confidence 

2.3.4  Family  closeness 

2.3.5  No  positives 

2.3.6  Other 
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2.1  How  ready  was  the  family? 

2.2  Resources  used  during 
deployment 

2.3  Positive  aspects  of 
deployment 

2.4  Negative  aspects  of 
deployment 


This  slide  demonstrates  how  a  theme  in  the  example  project  was  further  divided  into  sub¬ 
themes.  In  this  instance,  all  the  text  coded  as  positive  aspects  of  deployment  was  examined  by 
the  researchers,  who  inductively  determined  these  five  codes,  along  with  “other,”  to  represent 
the  subthemes  heard. 

The  other  substantive  codes  were  also  broken  into  subthemes  that  are  not  shown  on  this 

slide. 
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Developing  and  Using  a  Coding  Tree 

•  Initial  tree  is  codes  based  on  questions  and 
expected  themes 

•  Test  against  data 

*  Test  understanding  of  codes  within  research  team 

*  Revisit  coding  tree  as  necessary 

•  Test  reliability  of  coding  across  coders 

-  Roundtable 

-  Statistically  (kappa) 

*  Adjust  amount  of  double-coding  as  necessary 
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Coding  trees  evolve  during  a  project.  When  the  coding  tree  is  initially  designed,  the  codes  are 
often  deductive — based  on  the  questions  asked  and  the  expected  themes.  The  researchers  test 
the  initial  codes  against  the  data  to  see  whether  the  codes  can  he  applied  well,  and  whether 
the  codes  reflect  the  data.  It  is  important  to  ensure  that  the  research  team  members  doing  the 
coding  all  understand  the  codes,  and  the  application  of  the  codes,  the  same  way.  For  example, 
if  coding  both  “positive  examples”  and  “negative  examples”  of  an  issue,  how  positive  do  they 
have  to  be  for  that  code  to  apply?  What  if  they  are  not  either  enthusiastically  positive  or  nega¬ 
tive?  The  coding  tree  should  be  revisited  as  necessary  to  clarify  codes,  to  add  additional  codes 
as  themes  emerge  inductively,  to  combine  codes  that  do  not  offer  a  useful  distinction  from  one 
another,  and  to  eliminate  codes  that  are  not  useful. 

While  coding  is  most  efficient  when  only  a  single  researcher  codes  a  particular  passage, 
most  projects  begin  with  multiple  researchers  coding  the  same  passages  from  the  same  inter¬ 
views.  This  permits  an  assessment  of  whether  different  researchers  are  interpreting  and  apply¬ 
ing  the  codes  the  same.  There  are  two  different  ways  to  assess  the  reliability  of  coding  across 
coders.  One  includes  tabletop  meetings  or  roundtables  to  discuss  how  different  codes  are  being 
interpreted  and  applied.  The  project  team  might  review  a  single  session  coded  by  different 
members  of  the  project  team  and  discuss  why  different  researchers  did  (or  did  not)  code  the 
session  identically.  Most  coding  software  also  offers  the  opportunity  to  statistically  compare 
the  coding  of  different  coders.  This  is  a  complicated  process  that  produces  a  quantitative  value 
(kappa).  The  kappa  value  may  not  provide  a  clear  understanding  of  those  situations  where  two 
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coders  agree  that  a  statement  reflects  a  theme,  hut  coder  A  includes  more  context  for  the  com¬ 
ment  than  does  coder  B. 

Project  leaders  should  repeatedly  assess  the  reliability  of  the  coding  hy  having  every  nth 
session  douhle-coded  throughout  the  coding  effort.  When  coding  reliahility  is  high,  they  can 
increase  n.  When  coding  reliahility  is  of  concern,  n  needs  to  decrease  and  some  sessions  may 
need  to  he  recoded. 
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Common  Data  Analysis  Queries 


Intersect 

All  units  tagged  by  all  of  codes  given 

How  did  spouses  with  children  define  readiness? 

Union 

All  units  tagged  by  any  of  codes  given 

Did  the  same  types  of  spouses  tend  to  have  both 
negative  and  positive  comments?  Show  me  the 
negative  and  the  positive  aspects  together. 

Less 

All  coded  by  first  but  not  any  of  others  given 

Show  me  all  the  positive  aspects  (coded  at  2.3) 
that  have  not  been  coded  at  a  positive  subtheme. 

Just  One 

All  coded  by  only  one  of  those  given 

Show  me  any  comment  that  is  only  about 
financial  benefit  and  not  about  spouse 
independence  or  the  other  positives. 
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Data  are  coded  so  that  they  can  he  manipulated  with  queries  that  will  provide  the  researcher 
with  insights  about  the  data.  This  slide  represents  some  basic  analysis  queries  of  qualitative 
data,  using  coding  and  analysis  software. 

The  first  is  an  intersection.  Intersections  permit  the  researcher  to  inquire  about  any  text 
coded  by  more  than  one  code,  including  attribute  codes.  For  example,  an  intersection  query 
might  be  “How  did  all  the  spouses  who  are  parents  (attribute  code)  define  readiness  for  their 
family  (substantive  code)?” 

A  union  query  permits  the  researcher  to  see  whether  a  single  session  included  discussion 
of  two  different  types.  For  example,  did  the  same  interview  session  include  conversation  about 
positive  aspects  and  about  negative  aspects?  A  union  of  all  positive  and  all  negative  comments, 
in  output  format  by  session,  would  show  the  researcher  whether  sessions  with  robust  positive 
comments  also  had  robust  negative  comments. 

A  less  query  provides  the  researcher  with  the  opportunity  to  see  all  text  coded  by  one  code 
that  is  not  coded  by  another.  For  example,  if  a  researcher  wanted  to  ensure  that  all  the  com¬ 
ments  that  had  been  coded  as  positive  aspects  (2.3)  had  been  assigned  to  a  subtheme,  a  less 
query  would  highlight  those  that  had  not  been  assigned  to  a  subtheme. 

Sometimes  a  researcher  may  be  concerned  that  two  themes  are  intertwined,  and  in  these 
cases  2.  just  one  query  can  be  useful.  For  example,  if  the  study  found  that  spouses  mentioned 
that  they  felt  more  independent  and  self-confident,  might  that  be  related  to  the  financial  ben¬ 
efit  of  deployment?  If  that  were  so,  then  many  of  the  passages  that  mention  increased  inde¬ 
pendence  and  self-confidence  as  a  positive  aspect  of  deployment  (2.3.3)  would  also  mention 
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financial  benefit  (2.3.1).  This  type  of  query  could  permit  the  researcher  to  examine  those  com¬ 
ments  that  are  only  about  one  or  the  other  of  these  subthemes,  to  ensure  that  they  are  distinct 
issues. 
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Creating  an  Analysis  Matrix:  Intersection  of 
Parental  Status  and  Positive  Aspects 


With 

Children 

(N=100) 

No 

Children 

(N=100) 

Financial 

44 

69 

Patriotism/pride 

60 

20 

Independence/ 

self-confidence 

21 

62 

Family  closeness 

40 

37 

No  positives 

11 

20 

Not  actual  data 

RAND 


When  the  researcher  wants  to  determine  the  number  of  intersections  across  themes  and  suh- 
themes,  an  analysis  matrix  provides  a  count.  The  example  shown  above  determines  the  inter¬ 
sections  between  the  subthemes  of  positive  aspects  of  deployment  and  the  attribute  of  being 
a  parent  or  not  having  children.  There  were  100  parents  and  100  spouses  without  children 
included  in  this  mock  study,  and  this  matrix  indicates  that  44  parents  and  69  spouses  without 
children  mentioned  financial  benefit  as  a  positive  aspect  of  deployment,  that  patriotism  and 
pride  in  the  service  member  was  more  of  an  issue  for  families  with  children  than  spouses  with¬ 
out  children,  and  that  spouses  without  children  were  more  likely  to  note  independence  and 
increased  self-confidence  as  a  positive  aspect.  Spouses  without  children  were  also  almost  twice 
as  likely  (20  versus  1 1)  to  note  that  they  had  experienced  no  positive  aspects  of  deployment. 

Note  that  the  matrix  indicates  the  number  of  respondents  that  have  some  text  coded  at 
this  theme,  and  that  the  researcher  must  know  the  total  number  of  interviews  with  that  attri¬ 
bute  (in  this  case,  N=100  for  both  attributes)  to  comprehend  the  share  of  the  total  represented 
by  the  matrix  entries.  Some  software  permits  the  researcher  to  select  any  of  these  numbers  to 
see  the  actual  text  represented  by  the  number. 
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Creating  an  Analysis  Matrix:  Intersection  of 
Gender  and  Positive  Aspects 


Male 

(N=25) 

Female 

(N=175) 

Financial 

15 

98 

Patriotism/pride 

5 

75 

Independence/ 

self-confidence 

0 

83 

Family  closeness 

2 

75 

No  positives 

10 

21 

Not  actual  data 
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Here  is  another  example  of  an  output  matrix  assessing  the  same  substantive  suhthemes  (posi¬ 
tive  aspects)  against  a  different  attribute  (gender).  With  the  knowledge  that  there  were  25 
male  interviewees  and  175  female  interviewees,  this  matrix  output  permits  the  researcher  to 
assess  the  positive  aspects  that  were  salient  to  the  respondents  with  different  attributes.  In  this 
instance,  the  researcher  might,  for  example,  conclude  that  financial  benefits  were  noted  by  a 
similar  proportion  of  men  and  women  (15  of  25  and  98  of  175),  and  that  patriotism  and  pride, 
increased  independence  and  self-confidence,  and  family  closeness  were  more  likely  to  be  men¬ 
tioned  by  female  spouses  than  by  male  spouses.  Also,  male  spouses  were  more  likely  to  note 
that  they  had  not  experienced  any  positive  aspects  from  deployment. 
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Exercise  6 

•  Review  text  coding 

*  Identify  subthemes 


RAND 


In  Exercise  6,  the  final  exercise,  you  will  review  text  coding  and  identify  subthemes  for  sample 
focus  group  notes.  The  exercise  is  provided  on  pp.  135-138. 
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Overview 

•  Introduction  to  data  collection  methods 
'Applications  from  the  real  world 
•Semi-structured  interviews 

•  Focus  groups 

•  Brief  introduction  to  data  management  and  analysis 

•  Closing  remarks 


RAND 


This  course  concludes  with  the  following  points. 
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Reporting  Qualitative  Findings 

•  Summarize  patterns,  themes,  perspectives,  and 
perceptions 

•  Provide  rich  descriptions  and  quotations 

*  Unit  of  analysis  different 

-  Interview  -  individual 

-  Focus  group  -  group 

*  Extent  to  which  data  can  be  quantified  or  generalized 
varies 

-  Interviews  -  rigor  of  approach  and  sample  size  determine  whether  you 
can  quantify  or  generaiize  data 

-  Focus  groups  -  can  quantify  across  common  characteristics  of  groups 

-  Focus  groups  -  can  never  generalize  to  the  rest  of  the  population 


RAND 


The  merit  of  qualitative  research  is  tremendous.  One  can  summarize  patterns,  themes,  perspec¬ 
tives,  and  perceptions.  Qualitative  data  permits  researchers  to  provide  rich  descriptions  and 
quotations  unique  to  qualitative  research.  However,  it  is  important  that  researchers  remember 
that  the  unit  of  analysis  differs  for  interviews  and  focus  groups;  interview  data  represent  the 
comments  of  an  individual,  but  focus  group  data  can  only  indicate  the  views  of  the  entire 
group.  The  extent  to  which  the  data  can  be  either  quantified  or  generalized  to  the  broader  pop¬ 
ulation  also  varies.  With  interviews,  it  is  possible  to  generalize  the  data  if  the  research  incor¬ 
porated  sufficient  rigor  and  sample  size.  Focus  group  data  can  be  quantified  across  the  types  of 
groups  that  were  conducted,  based  on  the  common  characteristics.  However,  researchers  can 
never  generalize  focus  group  data  to  the  broader  population. 
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Course  Summary 

*  Rigorous  research  methods  result  in  high-quality 
research  with  credible  results 

*  Methods  can  be  combined  to  complement  one  another 

*  Interview  and  focus  group  approaches  should  consider: 

-  Sampling 

-  Question  design 

-  Protocol  development 

-  Staffing  and  training 

-  Moderating  techniques 

-  Data  capture 

*  Qualitative  data  should  be  systematically  analyzed 

*  Qualitative  data  can  provide  a  unique  depth  of 
understanding 

*  Important  not  to  overextend  implications  of  qualitative 

findings 

RAND 


High-quality  and  rigorous  research  methods  produce  high-quality  research  with  credible 
results.  There  are  a  variety  of  data  collection  methods,  and  many  of  them  complement  one 
another  well. 

This  course  has  provided  researchers  with  steps  for  thinking  about  the  process  of  qualita¬ 
tive  data  collection  and  some  tools  that  can  aid  them  in  collecting  data.  The  slide  reviews  some 
of  the  methodology  issues  we  have  covered  that  are  important  to  both  interviews  and  focus 
groups. 

Finally,  this  course  has  emphasized  that  qualitative  data  should  be  systematically  ana¬ 
lyzed  to  make  full  use  of  the  data  collected.  If  done  correctly,  the  analysis  provides  a  unique 
depth  of  understanding  of  the  topic  material.  However,  researchers  must  always  take  care  not 
to  overextend  the  implications  of  qualitative  findings,  and  they  must  always  state  clearly  the 
caveats  and  limitations  of  the  research. 


Exercise  1:  Interview  Probes 


Discussion 

Exercise  1  appears  on  the  following  page.  The  intent  of  this  exercise  is  to  consider  the  types 
of  participants  that  are  appropriate  to  address  a  research  question,  and  to  identify  prohes  that 
might  he  used  to  elicit  further  information  on  the  question  of  interest.  In  the  example  case, 
researchers  are  conducting  a  study  on  quality-of-life  issues  for  Alzheimer’s  Disease  patients  and 
hope  to  address  the  question.  How  does  Alzheimer’s  Disease  affect  day-to-day  life? 

First,  you  should  consider  all  possible  types  of  participants  who  might  address  these 
research  questions.  You  should  consider  the  characteristics  that  might  influence  how  people 
think  about,  feel  about,  or  experience  the  issues  you  want  to  discuss.  You  might  include  the 
following  possible  candidates  in  the  interviews: 

•  Alzheimer’s  patients 

•  Spouses 

•  Other  family  members 

•  Caretakers  (both  informal  and  formal) 

•  Doctors  or  other  medical  professionals 

Next,  you  need  to  determine  whether  there  are  any  important  subgroups  that  should  be 
included  in  the  research.  For  example,  is  it  important  to  create  subgroups  of  patients  based  on 
length  of  time  with  the  disease,  age,  or  extent  of  disease?  Might  there  be  important  differences 
to  investigate  across  these  different  subgroups? 

The  second  part  of  the  exercise  considers  the  probes  that  might  be  appropriate  for  the 
research  question.  These  probes  are  necessary  to  develop  the  interview  protocol.  Here  it  is 
important  to  keep  in  mind  the  different  participants.  Not  all  probes  will  be  appropriate  for 
each  respondent.  In  addition,  there  might  be  small  changes  to  probes  based  on  to  whom  the 
researchers  are  addressing  the  question.  Some  examples  of  probes  for  patients  might  include 

•  How  is  your  job  affected? 

•  How  is  your  social  life  affected? 

•  What  kinds  of  activities  are  harder  for  you  since  you  developed  Alzheimer’s  Disease? 
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Interview  Probes  for  Study  of  Alzheimer's  Disease 

For  a  study  on  quality-of-life  issues  for  Alzheimer’s  Disease  patients,  the  following  research 
questions  are  posed: 

•  What  are  the  physical  effects  of  Alzheimer’s  Disease? 

•  What  are  the  emotional  effects  of  Alzheimer’s  Disease? 

•  How  does  Alzheimer’s  Disease  affect  day-to-day  life? 

•  How  do  patients  cope  with  Alzheimer’s  Disease  and  the  problems  they  have  as  a  result 
of  Alzheimer’s  Disease? 

1.  First,  list  all  the  possible  types  of  participants  who  might  address  these  research 
questions.  Consider  what  characteristics  might  influence  how  people  think  about, 
feel  about,  or  experience  the  issues  you  want  to  discuss. 

2.  Identify  probes  for  the  question  in  boldface  type,  keeping  in  mind  the  different 
participants.  How  might  the  probes  differ  based  on  who  you  are  asking  to  address 
the  question? 


Exercise  2:  Interview  Protocol 


Discussion 

On  the  following  page  is  a  sample  interview  protocol.  The  protocol  includes  an  introduction 
and  closing  section  and  separates  the  questions  hy  topic. 

In  the  introduction,  the  interviewers  will  introduce  themselves  and  their  organization  to 
the  respondent.  They  will  provide  an  overview  of  the  study,  information  on  data  use,  and  any 
assurances  about  confidentiality.  They  will  also  offer  the  respondent  an  opportunity  to  ask 
questions  about  the  study. 

Next  are  the  research  questions.  Note  that  time  estimates  are  provided  for  each  section. 
As  mentioned  previously,  it  is  important  to  consider  the  amount  of  time  during  the  interview 
that  should  be  devoted  to  each  topic. 

The  sample  protocol  is  highlighted  where  improvements  can  be  suggested. 

Question  1  asks  the  respondent  about  two  different  topics — thus  it  is  a  double-barreled 
item,  which  should  be  avoided.  Here  it  would  be  best  to  reframe  this  item  into  two  separate 
items: 

•  To  begin,  please  describe  the  various  users  of  the  library. 

•  Tell  me  about  the  kinds  of  activities  for  which  people  use  the  library. 

Question  2  is  framed  as  a  “yes/no”  question  and  as  such  is  not  helpful  for  encouraging 
conversation.  It  might  be  rephrased  along  the  lines  of  “What  are  the  user  needs  that  are  not 
being  met?” 

As  with  Question  2,  Question  6d  is  a  “yes/no”  question  and  should  be  rephrased.  In  addi¬ 
tion,  the  term  “user  friendly”  could  cover  a  wide  variety  of  issues,  and  asked  as  such,  it  might  be 
unclear  to  the  researchers  what  the  respondent  is  considering  in  his  answer.  A  better  question 
might  read,  “How  is  the  library  layout  helpful  to  users?” 

Question  8  poses  a  question  about  the  use  of  library  services.  While  the  question  itself  is 
straightforward,  it  also  includes  a  series  of  examples  in  parentheses.  The  structure  of  this  ques¬ 
tion  may  lead  to  it  being  administered  differently  by  interviewers.  For  instance,  one  interviewer 
might  read  the  items,  while  another  might  not.  Thus,  the  interviewers  should  be  provided  with 
more  explicit  guidance  about  how  to  ask  this  question. 

There  are  two  possible  problems  with  Question  10.  As  with  Question  2,  Question  10  is 
a  “yes/no”  question  and  should  be  rephrased  if  it  is  administered.  For  instance,  “What  ser¬ 
vices  are  underutilized?”  However,  the  more  important  substantive  issue  with  Question  10  is 
whether  it  is  appropriate  to  the  respondent.  Is  the  researcher  interviewing  someone  that  is  likely 
to  have  factual  information  about  this  issue?  For  example,  this  question  might  be  appropriate 
to  library  administrative  staff,  but  it  would  not  be  appropriate  to  students. 
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State  University  Library  Interview  Protocol 


Introduction 

Thank  you  for  aqreeinq  to  meet  with  us.  I’m  from  the  RAND 

Corporation,  a  private,  non-profit  research  organization  in  Santa  Monica,  CA.  1  also  have 

(5  minutes) 

mv  colleaque  present  to  take  notes  for  us. 

We  are  speaking  with  students  and  library  staff  members  to  get  various  impressions  of 
the  University’s  library.  The  study  is  being  funded  by  State  University’s  Alumni 

Committee.  As  a  librarian,  we  would  like  to  talk  with  you  about  users’  needs,  the  facility, 
and  the  type  of  services  offered  to  patrons.  What  we  learn  from  today’s  discussion  will 
help  us  improve  the  library’s  facilities  and  services. 

We  will  treat  your  answers  as  confidential.  We  will  not  include  your  names  or  any  other 
information  that  could  identify  you  in  any  reports  we  write.  We  will  destroy  the  notes  and 
audiotapes  after  we  complete  our  study  and  publish  the  results. 

Do  you  have  any  questions  about  the  study? 

Topic  1 

Topic  #1:  Needs 

(10  minutes) 

1 .  To  begin,  please  describe  the  various  users  of  the  libraw  and  tell  me  about  th^ 
kinds  of  activities  for  which  the  librafi?  is  used.l 

a.  PROBE:  Tell  me  about  the  kinds  of  people  who  use  the  library. 

b.  PROBE:  Why  do  these  people  come  to  the  library? 

2.  Are  there  user  needs  that  are  not  being  met? 

Topic  2 

Topic  #2:  Faciiity 

(20  minutes) 

Now,  we’d  like  to  discuss  your  impressions  of  the  library’s  facilities. 

3.  What  words  would  you  use  to  describe  the  library  facilities? 

4.  How  do  the  facilities  meet  the  needs  of  the  community? 

5.  How  do  the  facilities  meet  staff  needs? 

6.  How  could  the  library  improve  its  facilities? 

a.  PROBE:  How  accessible  are  books,  journals  and  other  materials? 

b.  PROBE:  Tell  me  about  spaces  to  sit  and  do  work. 

c.  PROBE:  Tell  me  about  access  to  computers  and  other  technology. 

d.  PROBE:  Is  the  layout  user  friendly?! 

7.  If  you  could  design  the  perfect  library,  what  would  it  look  like? 

Topic  3 

Topic  #3:  Services 

(15  minutes) 

The  last  thing  that  I’d  like  to  discuss  with  you  are  the  services  that  the  library  offers. 

8.  What  library  services  are  used  the  most  often?  (borrowing  books,  reserving  books 
online,  interlibrary  loan,  research  classes)! 

9.  What  could  the  library  do  to  improve  these  services? 

10.  Are  there  services  that  are  underutilized?! 

1 1 .  What  additional  services  should  the  library  provide? 

Final  thoughts 

(5  minutes) 

Those  were  all  of  the  questions  that  we  wanted  to  ask. 

12.  Do  you  have  any  final  thoughts  about  the  library  that  you  would  like  to  share? 

Thank  you  for  your  time. 

Exercise  3:  Interview  Notes 


Discussion 

On  the  following  page  is  a  brief  subset  of  an  interview.  The  lefthand  section  indicates  the  text 
that  would  result  if  the  interview  were  taped  and  transcribed.  Thus,  this  is  the  most  accurate 
representation  of  the  verbal  response  of  the  interviewee.  The  middle  section  indicates  the  notes 
taken  during  the  interview.  The  righthand  section  indicates  a  subset  of  the  hnal  notes,  those 
that  would  correspond  with  this  portion  of  the  interview. 

There  are  several  worthwhile  observations  regarding  what  the  note-taker  has  written. 
First,  the  note-taker  is  writing  short  passages  representing  the  comments,  using  abbreviations. 
Also,  the  note-taker  is  capturing  nonverbal  aspects  of  the  interview,  such  as  where  the  speaker 
takes  a  long  pause  or  laughs,  and  also  where  the  speaker  seems  uncomfortable  answering  the 
question.  The  note-taker  also  indicates  where  the  notes  reflect  an  exact  quote. 

The  final  notes  on  the  right  side  are  not  the  same  as  the  transcribed  version.  They  provide 
a  less  detailed  account  of  the  verbal  interaction,  but  they  also  include  nonverbal  observations 
that  are  not  present  in  the  transcript.  The  final  notes  are  considerably  more  detailed  than  the 
original  notations.  They  are  written  in  complete  sentences,  without  abbreviations.  The  com¬ 
pleteness  of  this  passage  underscores  the  importance  of  writing  the  final  notes  as  soon  as  pos¬ 
sible  after  the  interview.  The  final  notes  also  indicate  who  is  speaking,  which  would  not  neces¬ 
sarily  be  present  in  the  transcript  text,  unless  the  speaker  identity  was  added. 
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Examples  of  Notes 


Participant  Says 

Note-Taker  Writes 

Types  in  Report 

"Oh,  1  heard  about  that 
new  exercise  facility  and 
it's  supposed  to  be  really 
nice.  But  I'd  have  to  get 
on  two  different  buses  to 
get  there,  which  is  such  a 
hassle.  Especially  taking 
that  red  Line  bus  .  .  .  that 
one's  never  on  time.  Last 
week  1  was  late  for  work 
twice  because  of  that  bus. 
Anyway  ...  1  know  1  should 
exercise,  but ...  1  guess 
.  .  .  exercising  at  a  gym  just 
isn't  for  me.  I'm  overweight 
and  1  feel  like  all  the  people 
who  go  to  exercise  at  a  gym 
are  already  in  shape  and 

I'd  feel  like  such  a  blimp 
surrounded  by  all  those 
buff  young  kids,  you  know? 
Maybe  if  they  had  exercise 
classes  for  people  like  me — 
people  who  haven't  really 
exercised  before  and  are 
just  getting  started.  Then 

1  might  do  that.  Because  1 
don't  know  how  to  use  the 
equipment  and  it  would 
be  good  if  there  was  an 
instructor  who  could  show 
me." 

•  knows  facility,  heard  it's 
nice 

•  problem — needs  two 
buses  (Red  Line  bad) 

•  [long  pause]  doesn't  like 
exercising  @  gym 

•  people  at  gyms  already  in 
shape 

•  would  "feel  like  such  a 
blimp  surrounded  by  all 
those  buff  young  kids" 
[laughs] 

•  might  go  CC  if  ex.  Class  or 
instructor  to  demo  eqpmt 

•  [UNCOMFORTABLE] 

R  knows  about  the  exercise 
facility  and  the  community 
center  and  says  that  she's 
heard  it's  nice.  R  doesn't 
use  the  facility  for  several 
reasons.  First,  R  reports 
that  transportation  is  a 
problem — she  needs  to 
take  two  buses  to  get  to 
the  community  center, 
and  one  of  these  buses  is 
especially  unreliable.  [Long 
pause.]  R  also  reports  that 
she  doesn't  like  exercising 
at  a  gym.  R  thinks  that  the 
people  who  go  to  gyms 
are  already  in  shape  and 
she  would  "feel  like  such 
a  blimp  surrounded  by  all 
those  buff  young  kids" 
[laughs].  R  said  that  she 
might  go  to  the  community 
center  to  exercise  if  they 
offered  exercise  classes 
for  beginners  and  had 
instructors  to  demonstrate 
how  to  use  the  equipment. 

[R  SEEMED 

UNCOMFORTABLE 
ANSWERING  THIS 

OUESTION.] 

Exercise  4:  Focus  Group  Protocol 


Discussion 

This  exercise  is  structured  similarly  to  Exercise  2,  but  provides  an  example  of  a  focus  group 
protocol  (on  the  following  two  pages).  Like  the  interview  protocol,  the  focus  group  protocol 
includes  an  introduction  and  closing  section  and  separates  the  questions  by  topic.  Here  you 
also  see  a  large  section  of  “ground  rules.”  The  ground  rules  are  read  by  the  moderator  to  the 
entire  group.  The  ground  rules  cover  various  issues  related  to  informed  consent,  and  they  pro¬ 
vide  more  detail  to  participants  on  the  expectations  for  the  group  interaction.  This  can  be  espe¬ 
cially  helpful  for  participants  who  are  unfamiliar  with  the  focus  group  process. 

This  focus  group  protocol  would  benefit  from  some  improvements. 

One  consideration  is  the  amount  of  time  dedicated  to  each  topic  area.  In  the  example. 
Topic  1  receives  25  minutes  of  discussion  time.  Topic  3  receives  30  minutes,  and  Topic  2 
receives  only  10  minutes.  Given  the  number  of  questions  posed  for  Topic  2,  it  seems  unlikely 
that  the  10  minutes  allocated  would  be  adequate  time  to  discuss  this  issue. 

Looking  at  the  individual  questions,  we  see  that  Question  2  is  problematic.  It  includes 
multiple  questions,  the  first  of  which  is  a  yes/no  question  and  the  second  of  which  is  double- 
barreled.  This  question  should  be  rephrased  as  several  individual  questions,  with  some  reword¬ 
ing  to  prevent  the  questions  from  being  too  vague.  Resulting  questions  might  include: 

•  What  club  meetings  or  study  groups  have  you  held  in  the  library? 

•  Where  did  you  hold  these  meetings? 

•  Tell  me  how  this  space  worked  out  for  this  purpose. 

Question  4  is  a  yes/no  question,  and  it  is  also  vague.  The  researcher  needs  to  specify  the 
needs  of  interest. 

Question  6  improves  on  the  similar  question  asked  in  the  interview  protocol  presented  in 
Exercise  2.  No  longer  are  the  examples  shown  in  parenthesis,  but  instead  are  presented  here  as 
specific  probes. 

Finally,  note  that  Question  10  is  an  example  of  a  “big-picture”  question. 
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State  University  Library  Focus  Group  Protocol 


1.  Welcome 

Welcome.  I  want  to  thank  you  for  coming  today.  My  name  is  and  I  will  be 

the  facilitator  for  today’s  group  discussion.  1  am  a  researcher  and  1  work  for  the  RAND 
Corporation,  a  priyate,  non-profit  research  organization  in  Santa  Monica,  CA.  We  also  haye 
present  to  take  notes  for  us. 

We  inyited  you  to  take  part  in  this  group  discussion  today  because  you  are  all  students  at  State 
Uniyersity.  We  would  like  to  talk  with  you  today  about  your  impressions  of  the  Uniyersity’s 
library. 

The  study  is  funded  by  the  State  Uniyersity’s  Alumni  Committee.  What  we  learn  from  today’s 
discussion  will  help  us  improye  the  library’s  facilities  and  services. 

2.  Ground  Rules 

Before  we  begin,  1  would  like  to  reyiew  a  few  ground  rules  for  the  discussion. 

a.  1  am  going  to  ask  you  seyeral  questions;  we  do  not  haye  to  go  in  any  particular  order 
but  we  do  want  eyeryone  to  take  part  in  the  discussion.  We  ask  that  only  one  person 
speak  at  a  time. 

b.  Feel  free  to  treat  this  as  a  discussion  and  respond  to  what  others  are  saying,  whether 
you  agree  or  disagree.  We’re  interested  in  your  opinions  and  whateyer  you  haye  to 
say  is  fine  with  us.  There  are  no  right  or  wrong  answers.  We  are  just  asking  for  your 
opinions  based  on  your  own  personal  experience.  We  are  here  to  learn  from  you. 

c.  Don’t  worry  about  haying  a  different  opinion  than  someone  else.  But  please  do 
respect  each  other’s  answers  or  opinions. 

d.  If  there  is  a  particular  question  you  don’t  want  to  answer,  you  don’t  haye  to. 

e.  We  will  treat  your  answers  as  confidential.  We  are  not  going  to  ask  for  anything  that 
could  identify  you  and  we  are  only  going  to  use  first  names  during  the  discussion.  We 
also  ask  that  each  of  you  respect  the  priyacy  of  eyeryone  in  the  room  and  not  share  or 
repeat  what  is  said  here  in  any  way  that  could  identify  anyone  in  this  room. 

f  We  are  tape  recording  the  discussion  today  and  also  taking  notes  because  we  don’t 
want  to  miss  any  of  your  comments.  Howeyer,  once  we  start  the  tape  recorder  we  will 
not  use  anyone’s  full  name  and  we  ask  that  you  do  the  same.  Is  eyeryone  OK  with  this 
session  being  tape  recorded?  [GET  VERBAL  CONSENT  TO  TAPE  RECORD 
DISCUSSION.  IF  A  PARTICIPANT  DECIDES  THAT  S/HE  DOES  NOT  WANT 

TO  BE  TAPED  AND  WANTS  TO  LEAVE,  S/HE  SHOULD  STILL  BE  PAID  THE 
FULL  AMOUNT] 

g.  We  will  not  include  your  names  or  any  other  information  that  could  identify  you  in 
any  reports  we  write.  We  will  destroy  the  notes  and  audiotapes  after  we  complete  our 
study  and  publish  the  results. 

h.  Finally,  this  discussion  is  going  to  take  about  two  hours  and  we  ask  that  you  stay  for 
the  entire  meeting.  At  the  end  of  the  discussion  we  will  giye  you  $50  to  thank  you  for 
participating. 

Does  anyone  have  any  questions  before  we  start? 

3.  Introductions 

[START  TAPE  RECORDER  NOW.] 

(5  minutes) 

I’d  like  to  go  around  the  table  starting  on  my  right  and  haye  each  person  introduce  him  or 
herself.  Please  tell  us  your  first  name  only  and  tell  us  what  year  you  are  in  and  your  major. 

(continued  on  next  page) 
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4.  Group  Discussion 
-  Topic  1 

Topic  #1:  Needs 

The  first  thing  that  we’d  like  for  you  to  discuss  is  the  main  activities  that  you  use  the  library 

(25  minutes) 

for. 

1 .  To  start  off,  tell  us  about  your  average  trip  to  the  library. 

a.  PROBE:  What  activities  do  you  use  the  library  for? 

b.  PROBE:  How  long  does  your  visit  usually  last? 

c.  PROBE:  What  sections  of  the  library  do  you  study  in? 

2.  Have  you  ever  held  a  club  meeting  or  a  study  group  in  the  library?  Where  did  you 
hold  it  and  did  it  work  out  well? 

5.  Group  Discussion 
-  Topic  2 
(10  minutes) 

Topic  #2:  Facility 

Now,  we’d  like  to  discuss  your  impressions  of  the  library’s  facilities. 

3.  What  words  would  you  use  to  describe  the  library  facilities? 

4.  Are  the  facilities  meeting  your  needs? 

5.  How  could  the  library  improve  its  facilities? 

a.  PROBE:  How  accessible  are  books,  journals,  and  other  materials? 

b.  PROBE:  Tell  me  about  the  spaces  to  sit  and  do  work. 

c.  PROBE:  Tell  me  about  access  to  computers  and  other  technology 

d.  PROBE:  How  could  the  organization  of  the  library  be  improved? 

6.  Group  Discussion 
-  Topic  3 
(30  minutes) 

Topic  #3:  Services 

The  last  thing  that  I’d  like  to  discuss  with  you  today  is  your  impressions  of  the  services  that 
the  library  offers. 

6.  What  library  services  do  you  use  the  most  often? 

a.  PROBE:  How  often  do  you  borrow  books? 

b.  PROBE:  How  often  do  you  reserve  books  online? 

c.  PROBE:  How  often  do  you  use  interlibrary  loan? 

d.  PROBE:  What  other  service  have  you  used  often? 

7.  What  could  the  library  do  to  improve  these  services? 

8.  Why  haven’t  you  used  some  of  the  other  services  that  the  library  provides? 

9.  What  additional  services  would  you  like  for  the  library  to  provide? 

10.  If  you  had  a  chance  to  speak  with  the  president  of  the  university,  what  is  the  one  thing 
you  would  want  to  tell  her  about  the  library? 

7.  Final  thoughts 
(5  minutes) 

Those  were  all  of  the  questions  that  I  wanted  to  ask. 

11.  Does  anyone  have  any  final  thoughts  about  the  library  that  they  haven’t  gotten  to 
share  yet? 

8.  Review  and 
Wrap-up 
(5  minutes) 

Thank  you  for  coming  today  and  for  sharing  your  opinions  with  us.  We  hope  you  enjoyed  the 
discussion  today.  I’m  going  to  be  handing  out  the  payments.  [HAND  OUT  PAYMENTS.] 

Exercise  5:  Focus  Group  Notes 


Discussion 

This  exercise  is  structured  similarly  to  Exercise  3,  but  this  exercise  addresses  focus  group  notes. 
The  text  on  the  left  side  of  the  page  is  a  mock  transcript  of  a  focus  group.  The  right  side  of  the 
page  represents  the  handwritten  notes  taken  during  the  session.  The  final  written  notes  are 
included  after  the  transcript. 

The  focus  group  begins  with  the  moderator  asking  a  background  question.  Note  that  the 
note-taker  captures  the  data,  with  the  exception  of  individual  names,  which  are  not  included  in 
the  notes.  On  one  occasion,  when  recording  the  major  of  participant  3,  the  note-taker’s  notes 
are  insufficient.  The  participant  indicates  that  she  is  a  double  major  in  psychology  and  political 
science.  While  the  note-taker  includes  both  of  those  disciplines  in  the  notes,  it  is  not  clear  from 
the  notes  whether  the  student  is  a  double  major,  is  majoring  in  one  and  pursuing  a  minor  in 
the  other,  or  is  undecided  between  the  two  fields  of  study. 

The  next  page  of  notes  begins  with  a  substantive  question,  and  the  first  response  is  insuf¬ 
ficiently  complete.  The  transcript  indicates  how  the  moderator  probed  for  clarification.  The 
notes  on  this  page  indicate  how  the  note-taker  associates  each  comment  with  an  individual 
and  abbreviates  the  responses.  The  notes  also  indicate  a  mistake  made  by  the  note-taker  with 
the  use  of  “it.”  Pronouns  such  as  “it”  are  problematic  because  it  is  often  difficult  to  later  inter¬ 
pret  the  meaning.  In  this  instance,  “it”  referred  to  books,  and  “book”  should  have  been  used 
in  the  notes. 

At  the  bottom  of  this  page  of  transcription,  the  moderator  makes  an  error.  Two  partici¬ 
pants  (PI  and  P2)  have  indicated  that  they  use  the  library  just  for  checking  out  books.  The 
moderator  is  interested  in  a  handcount  of  others  who  have  a  similar  experience,  which  is  an 
acceptable  tool  during  a  focus  group.  However,  the  moderator  errs  with  her  introduction  to 
the  handcount,  when  she  states  that  “It  sounds  as  though  lots  of  you  use  the  library  just  for 
getting  books.”  In  fact,  only  two  participants  had  stated  that  theme,  and  the  moderator  has 
potentially  biased  this  portion  of  the  discussion  by  stating  this  theme  more  strongly  than  the 
data  permit.  The  moderator  should  have  said,  “I  understand  that  two  of  you  use  the  library 
only  for  checking  out  books. . .”  By  using  “two”  or  “a  couple,”  the  moderator  would  avoid  bias¬ 
ing  other  participants. 

The  beginning  of  the  third  page  of  the  transcript  begins  with  a  double-barreled  question. 
If  the  moderator  is  interested  in  asking  a  grand  tour  question  and  initiating  a  broad  descrip¬ 
tion  of  the  participants’  library  experiences,  this  might  be  appropriate.  However,  if  the  research 
wants  clear  answers  to  each  part  of  that  question,  the  question  is  misstated. 


127 


128  Data  Collection  Methods:  Semi-Structured  Interviews  and  Focus  Groups 


Soon  after  the  double-barreled  question,  the  participants  begin  to  speak  at  the  same  time. 
This  is  reflected  in  the  notes  on  the  right  side  of  the  page.  The  moderator  responds  to  this 
confusion  with  a  probe  to  obtain  control  of  the  discussion  and  prompt  a  single  individual  to 
speak. 

Another  item  highlighted  in  the  notes  for  the  purpose  of  this  exercise  is  three  asterisks, 
used  in  this  case  as  a  convention  to  indicate  that  “campus”  is  an  incomplete  reflection  of  the 
comment  made.  Projects  should  predetermine  conventions  that  will  be  understandable  to  the 
entire  research  team.  An  easily  found  convention  to  indicate  missed  discussion  permits  the 
note-taker  to  follow-up  with  the  participants  to  fill  those  gaps  when  the  moderator  provides 
that  opportunity. 

Later  in  this  page  of  transcript,  there  is  a  negative  interaction  between  two  of  the  par¬ 
ticipants  (P4  and  P7).  The  notes  indicate  that  there  is  also  a  physical  interaction,  and  that  P7 
has  held  up  a  handout  in  a  mocking  way  as  she  has  spoken.  Note  that  the  nonverbal  action  is 
not  captured  with  a  transcript  alone.  The  moderator  interjects  herself  into  this  interaction  to 
remind  the  participants  of  the  ground  rule  of  being  respectful  to  one  another  and  to  redirect 
the  discussion. 

This  interaction  is  followed  by  a  vague  response  from  P2  that  requires  the  moderator  to 
probe  for  clarification. 

The  second  portion  of  this  exercise  provides  the  final  written  notes  that  would  reflect  this 
same  focus  group  session.  They  begin  with  the  session  information  and  the  impressions  com¬ 
ments  that  were  noted  by  the  moderator  after  the  conclusion  of  the  session.  The  remainder  of 
the  notes  reflect  the  conversation  that  occurred  during  the  focus  group,  represented  with  com¬ 
plete  sentences,  associated  with  each  speaker,  in  the  order  in  which  the  comments  were  made. 
Nonverbal  aspects  of  the  focus  group,  such  as  indication  of  other  participants,  laughter,  volume 
of  voice,  and  descriptions  of  the  interactions,  are  included  in  brackets  to  clearly  indicate  that 
they  are  contextual  and  editorial  comments. 
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Excerpts  of  State  University  Library  Focus  Group  Transcript  and  Handwritten 
Notes 


Transcript 

Handwritten  Notes 

M: 

I’d  like  to  go  around  the  tabie  and  have  each 

8  parts: 

person  introduce  himself.  Please  tell  us  your  first 

1  female,  Jun,  biol 

name,  what  year  you  are,  and  your  major. 

2  male,  soph,  undec. 

P1 

My  name  is  Becky.  I’m  a  Junior  and  I’m  majoring 

3  female,  sen,  psych./poly  sci 

in  biology. 

4  female,  lib.  sci  -  Jun 

P2 

And  I’m  Mark.  I’m  a  sophomore  and  I  actually 

5  male,  sen  -  fin 

haven’t  decided  on  a  major  Just  yet. 

6  male,  sen  -  chem 

P3 

Hi,  I’m  Ashley.  I’m  a  senior.  And  I’m  double 

7  female,  Jun  -  art  hist 

majoring  in  psychology  and  political  science. 

8  male,  accntng  -  soph 

P2 

So  you  try  to  figure  out  how  politicians  think, 

huh?  Good  luck  with  that! 

M: 

And  who’s  this  next  to  Ashley? 

P4 

I’m  Carrie.  I’m  a  library  science  major. 

M: 

And  what  year  are  you  in  school  Carrie? 

P4 

Oh,  I’m  a  Junior. 

P5 

Uh,  my  name  is  Dan.  I’m  a  senior  and  I’m 

majoring  in  finance. 

P6 

Robert.  Also  a  senior.  Chemistry. 

P7 

Hi,  I’m  Hannah.  And  I’m  a  Junior  and  an  art 

history  major. 

M: 

And  last  but  not  least.... 

P8 

I’m  Eric.  And  I’m  majoring  in  accounting.  Oh  and 

a  sophomore. 

M: 

Great.  Thanks  for  those  introductions  everyone. 

M  -  What  is  avg.  library  trip? 

Well  we’re  here  to  talk  about  the  library.  So  to 

start  off  our  discussion,  could  you  tell  me  about 

what  an  average  trip  to  the  library  is  like  for  you? 

P1 

Short. 

1  -  Short, 

IM: 

What  do  vou  mean  bw  tharti 

Clarification 

M-  mean? 

P1 

I  don’t  stay  very  long. 

1  -  don’t  stay  long. 

M: 

Why  is  that?  What  are  you  going  to  the  library  to 

M  -  What  are  u  going  there  to  do? 

do? 

(continued  on  next  page) 
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P1 :  I  don’t  really  go  to  the  library  very  often.  Usually  I 

only  go  if  I  need  to  take  out  a  book.  But  then  I’m 
just  in  and  out  really  fast.  I  get  the  book  and  go.  I 
don’t  hang  out  there  or  do  work  there. 

M:  And  so  overall,  how  long  does  one  of  these  trips 

usually  last? 

P1 :  Just  a  few  minutes.  It  depends  on  if  the  book 

that  I  need  is  buried  in  a  far  corner  of  the  library 
or  if  it’s  easy  to  find.  But  as  soon  as  I  find  the 
book  and  check  out,  I  leave. 

M:  And  what  about  everyone  else?  What  are  your 

trips  to  the  library  like? 

P2:  I’m  the  same  as  Becky.  I  don’t  go  very  often  or 

stay  very  long.  Just  go  to  get  books.  Or 
sometimes  if  I’m  passing  by  and  need  to  use  the 
bathroom.  [LAUGHS] 

M:  It  sounds  as  though  lots  of  ipou  use  the  libraraj 

lust  for  Betting  books.  But  just  to  make  sure  rairt 

Ifour  hands  for  me.|  How  many  people  are  like 
Becky  and  Mark  and  just  use  the  library  to  take 
out  books?  Ok,  Robert  has  his  hand  up.  How 
about  the  rest  of  you?  What  are  your  visits  to  the 
library  like? 

P5:  I  usually  study  in  the  library.  I  can’t  study  in  my 

dorm  room  because  my  roommate  is  always 
playing  video  games  or  playing  guitar.  It’s  really 
distracting.  So  I  go  to  the  library  a  lot  to  get 
some  quiet. 

HlI  So  when  fou  Mo  to  the  libraff,  what’s  a  usual  trij 

like?  How  long  do  you  usually  stay? 

P8:  It  depends.  I  usually  stay  somewhere  between 

10  minutes  and  10  days.  Cause  getting  books 
can  be  fast  but  during  finals  week... I  mean,  I  live 
in  there  during  finals  week  so... 


1-  Don’t  go  often.  Go  to  take  out  bk.  In  + 
out  fast.  Don’t  hang  out/wrk  there. 


M  -  How  long  trip  last? 

1  -  Just  few  mins.  Depends  if  it’s  buried. 
When  I  find  it  -  check  out  +  leave. 


M  -  Evryl?  What  are  trips  like? 

2-  Same  as  PI .  Not  often/stay  long.  Just 
get  bks.  Or  use  bthrm.  [Ighs] 


Assumption  made — biasing — should  note  that 
it’s  2  who  make  that  comment,  not  “a  lot” 


M  -  Handcnt  -  #  just  take  out  bks?  =4  (P6, 
PI,  P2,  P8) 

M  -  Othrs  -  visits  like? 


5  -  Study.  Can’t  in  dorm  rm  -  rmmate 
playing  vid.  games/  guitar. 
Distracting.  Go  4  quiet. 


M  -  Usu.  trip/length  stay? 


Double  Barrel 


8  -  Depends,  stay  betw  10  mins  - 10  dys. 
Getting  books=fast,  but  live  there 
@  fins  wk 


(continued  on  next  page) 
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P7;  Yeah,  during  exam  week  it’s  crazy  in  there.  It’s 

like  the  whole  campus  is... 

7  -  exm  wk  crazy  -  campus  ***? 

P4:  Everyone  is  there  during  finals  week  and  it  gets 

to  be.... 

P2:  That’s  why  1  just  don’t  [INAUDIBLE]... 

P7:  ...trying  to  study  and  write  papers. 

P2:  . .  .too  much  of  a  hassie 

P6:  Yeah,  that’s  so  true! 

P4:  ...even  in  the  quiet  room. 

4  -  Evry1  @  fins  wk  studying 

[ALL  SPEAKING  AT  SAME  TIME] 

BEEMS  to  be  about  exm  WKi] 

SEEMS  TO  BE  CONSENSUS 

[THAT  PROBLEMS  AT  LIBRARYl 

THAT  WEEK] 

:M:  Ok,  so  I’m  having  trouble  hearing  evenfon^ 

Get  control 

because  there’s  a  lot  of  talking  going  on  here.  So 

M  -  lib  crowded  @  fins  (SPK  1  @  TIME) 

tt  sounds  like  during  finals  week,  the  librai¥  W 

7  -  Evry1  in  lib  during  fin.  wk. 

M  -  Problem? 

4  -  Annoying  -  too  loud.  More  party  than 

lib. 

7-  It’s  fun,  2.  Like  atmosph.  -  struggling 

together/helping  ea.  othr 

4-  Ridiculous.  UR  1  of  people  I  want  to 

scream  at  -  so  frustrating  -  not 

what  lib  for,  should  hang  out  in 

stud.  cntr.  (ANGRY) 

Venf  crowded.  I’d  like  to  hear  about  the  groblemal 

lhatifou’ve  encountered  in  the  libranf  during 

flnais  week,  but  we  need  to  speak  one  at  a  timil 

so  1  can  get  everything  you’re  saying.  Hannah,  1 

think  you  were  saying... 

P7:  Just  that  everyone  on  campus  is  in  the  library 

during  finais  week. 

M:  And  is  that  a  problem?  Do  you  have  trouble... 

P4:  It’s  annoying  that  it  gets  so  loud.  It’s  more  of  a 

party  than  a  library. 

P7:  But  it  can  be  kind  of  fun  too.  1  like  the 

atmosphere  during  finals  week.  Everyone’s  sort 

of... you  know,  kids  are  just  struggling  together 

and  helping  each  other  with  final  reports  and.... 

|P4:  That’s  so  ridiculous.  You’re  probably  one  of  thd 

people  that  1  just  want  to  scream  at  during  finals 

week  because...] 

P7;  Wow,  way  to  get  worked  up  over  the  library..  .| 

P4:  ...it’s  so  frustrating.  That’s  not  what  the  library  ii 

7  -  “Way  to  get  wrked  up  ovr  the  lib....” 

for.  If  you  want  to  hang  out  with  your  friends  then 

IHOLD  HANDOUT  LIKE  STOP -I 

do  it  in  the  student  center] 

IMOCKINGI 

(continued  on  next  page) 
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M:  I  want  to  stop  for  a  second  and  remind  everyone 

ihat  it’s  OK  to  have  different  opinions  and  we  del 

tifant  to  hear  what  eveiifone  has  to  say.  But  Sfotj 

heed  to  be  resaactful  of  one  another.  Mark,  yoij 

had  mentioned  something  earlier  when  we  weral 

hiking  about  finals  week.  What  were  fouj 

thoughts? 

P2:  Just  that  it’s  too  much  of  a  hassle  to  even  go  to 

the  library  during  finais  week. 

M:  What  do  fou  mean  by  a  hassle^  Clarification 


M  -  Diff  opins=OK,  repect  ea.  other  -  P2 
your  thghts? 

Redirect  and  reminder  of 
ground  rules 


2-  2  much  hassle  to  go  @  fins  wk. 
M-  mean  by  hassle? 


P2:  You  know... it’s  impossibie  to  find  anywhere  to 

sit.  And  it  can  be  realiy  noisy. 


2-  imposs.  2  find  seat.  Reaily  noisy. 
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Excerpt  of  State  University  Library  Final  Written  Notes 


Session  47 

October  23,  2007 

Sheila  Wilson,  moderator 

Robert  Smith,  note-taker 

Session  conducted  in  room  in  student  center.  8  participants.  1  did  not  show  and  1  arrived  25 

minutes  late  and  was  not  permitted  to  enter.  In  general,  the  discussion  proceeded  well  and 
everyone  participated,  although  they  did  have  times  when  they  all  spoke  at  once  and  there 
was  one  participant  that  was  not  always  respectful  of  others,  and  so  the  others  became  very 
irritated  with  her.  One  of  the  main  themes  appears  to  be  the  degree  to  which  the  library  is 
crowded  and  thus  unable  to  handle  demand  during  finals  week. 

1  female,  junior,  biology 

2  male,  sophomore,  undecided  major 

3  female,  senior,  double  major  in  psychology  and  political  science 

4  female,  junior,  library  science 

5  male,  senior,  finance 

6  male,  senior,  chemistry 

7  female,  junior,  art  history 

8  male,  accounting,  sophomore 

M  What  is  average  trip  to  the  library  like? 

1  Short. 

M  What  do  you  mean  by  that? 

1  I  don’t  stay  long. 

M  What  are  you  going  there  to  do? 

1  I  don’t  go  often.  I  go  to  take  out  books  so  I’m  in  and  out  of  the  library  fast.  Don’t  hang  out  or  work 
there. 

M  How  long  does  your  trip  usually  last? 

1  Just  a  few  minutes.  It  depends  if  my  book  is  buried  [difficult  to  find].  When  I  find  the  book,  I 
check  it  out  and  leave. 


(continued  on  next  page) 
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M  What  about  everyone  else?  What  are  your  trips  to  the  library  like? 

2  Same  as  [P1].  I  don’t  go  very  often  and  I  don’t  stay  long.  I  just  go  to  get  books,  or  to  use  the 
bathroom,  [laughs] 

M  I’d  like  to  take  a  handcount.  Please  raise  your  hands  for  me.  How  many  people  are  like  [P1  and 
P2]  and  use  the  library  just  to  take  out  books?  [4  answered  affirmatively:  P6,  P1,  P2,  P8] 

M  What  about  you  others;  what  are  your  visits  to  the  library  like? 

5  I  go  to  the  library  to  study.  I  can’t  study  in  my  dorm  room  because  my  roommate  is  always 

playing  video  games  or  his  guitar,  so  it’s  too  distracting  there.  So  I  go  to  the  library  because 
it’s  quiet  there. 

M  So  what  is  a  usual  trip  to  the  library  like  for  you?  How  long  do  you  usually  stay? 

8  It  depends.  I  can  stay  at  the  library  anywhere  from  10  minutes  to  10  days.  Because  it  can  be  fast 
to  just  get  a  book,  but  during  finals  week  I  live  at  the  library 

7  During  exam  week  it  gets  crazy  at  the  library. 

[EVERYONE  STARTED  TALKED  AT  ONCE,  BUT  THE  CONVERSATION  WAS  GENERALLY 
CONSENSUS  AGREEMENT  THAT  THE  LIBRARY  IS  “CRAZY”  DURING  EXAM  WEEK] 

M  [SPEAKS  LOUDLY]  OK,  so  I’m  hearing  that  the  library  is  crowded  during  exam  week.  I’d  like 
everyone  to  please  speak  one  at  a  time  so  we  can  hear  everyone. 

7  It  seems  like  everyone  is  in  the  library  during  finals  week. 

M  Is  that  a  problem? 

4  It’s  annoying  because  it’s  too  loud.  It’s  more  like  a  party  than  a  library. 

7  But  it’s  fun,  too.  I  like  the  atmosphere  at  the  library  during  finals  week.  People  are  struggling 
together  and  helping  each  other. 

4  [SEEMED  ANGRY]  That’s  ridiculous.  You’re  probably  one  of  those  people  I  want  to  scream  at. 
It’s  so  frustrating  because  that’s  not  what  the  library  is  for.  If  you  want  to  do  that  you  should 
hang  out  in  the  student  center. 

7  [MADE  FUN  OF  HER  FOR  GETTING  ANGRY  ABOUT  THE  LIBRARY] 

M  OK,  remember  that  it’s  ok  to  have  different  opinions.  Please  remember  to  be  respectful  of  each 
other.  [P2]  did  you  have  other  thoughts  about  the  library  during  finals  week? 

2  It’s  just  too  much  of  a  hassle  to  go  to  the  library  during  finals  week. 

M  What  do  you  mean  by  it  being  a  hassle? 

2  Just  that  it’s  impossible  to  find  a  seat,  and  it’s  really  noisy. 
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Discussion 

On  the  following  three  pages  are  a  brief  portion  of  focus  group  notes.  The  portion  provided 
here  is  a  mock  excerpt  of  a  focus  group  session  with  spouses  of  reservists  who  are  currently 
deployed.  The  moderator  has  just  asked,  “What  are  the  negative  aspects  of  having  your  service 
member  deployed?”;  the  participants’  responses  follow. 

The  second  version  of  the  excerpt  uses  color  to  indicate  how  the  responses  would  be 
coded  at  a  simple  level — into  either  negative  aspects  or  positive  aspects,  which  were  two  of  the 
substantive  codes  shown  in  the  slides.  Although  the  question  was  about  negative  aspects  of 
deployment,  the  first  response  (P2)  provides  a  comment  about  the  additional  deployment  pay 
as  a  positive.  The  next  five  comments  do  address  negative  aspects.  Then  the  moderator  sums 
the  answers  and  probes  for  additional  comments  about  negative  aspects.  The  responses  include 
one  negative  aspect  and  then  another  positive  aspect. 

The  third  version  of  the  excerpt  identifies  and  codes  subthemes  from  this  same  passage. 
Just  as  the  slides  provide  the  subthemes  that  had  been  discerned  from  the  data  regarding 
positive  aspects  of  their  occupation,  this  exercise  identifies  the  subthemes  regarding  negative 
aspects  of  the  occupation.  The  first  subtheme  is  financial  gain  as  a  positive  aspect,  and  slides 
indicate  that  the  code  number  for  that  subtheme  is  2.3.1.  The  first  negative  theme  observed  in 
the  subsequent  comment  is  the  issue  of  the  effect  of  deployment  on  children.  Thus  the  first  two 
negative  comments  (by  P3  and  P4)  are  coded  at  a  new  code  representing  this  subtheme,  2.4.1, 
Children’s  Issues.  The  next  three  comments  (by  P8,  PI,  and  P4)  are  comments  about  the  dif¬ 
ficulty  managing  household  responsibilities,  including  chauffeuring  children  to  activities  and 
dealing  with  childcare  issues.  These  are  coded  at  a  subtheme  code  2.4.2,  Household  Respon¬ 
sibilities.  The  moderator’s  probe  is  followed  by  a  comment  about  financial  issues  as  a  negative 
aspect  of  deployment.  Thus,  these  are  coded  as  a  code  for  financial  issues  as  a  negative  aspect, 
2.4.3  Financial.  Note  that  this  is  a  different  node  than  that  used  for  financial  issues  that  are 
positive  aspects  of  deployment  (2.3.1).  The  final  comment,  and  the  note  that  indicates  agree¬ 
ment  among  other  members  of  the  focus  group,  again  pertain  to  financial  issues  as  a  positive 
aspect,  so  they  are  coded  at  2.3.1,  Financial. 
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Excerpt  from  a  Focus  Group  of  8  Spouses  of  Deployed  Reservists 


M  We’ve  been  talking  about  the  positive  aspects  of  deployment,  what  are  the  negative 
aspects  of  having  your  service  member  deployed? 

2  Well,  obviously  we  miss  him  when  he’s  gone,  but  we’re  doing  fine,  and  the  additional 
pay  is  really  nice. 

3  I’m  concerned  about  how  much  the  kids  miss  him.  They  don’t  talk  about  it  much,  but 
my  younger  son  has  been  acting  up  in  school  since  his  dad  left. 

4  I’m  concerned  about  my  boys  as  well.  They’re  not  old  enough  to  understand  why  he’s 
gone,  but  they’re  upset  that  he’s  not  there. 

8  My  kids  are  teenagers,  and  so  they’re  old  enough  to  understand.  For  our  house,  it’s 
more  an  issue  of  logistics.  It’s  just  really  hard  for  me  to  get  my  three  kids  to  three 
different  sets  of  activities  when  I’m  the  only  driver.  Their  friends’  parents  help,  but 
sometimes  my  kids  can’t  do  all  the  things  they  want  to  do  because  I  simply  can’t  get 
them  there. 

1  That’s  the  problem  in  our  family  as  well.  My  daughters  have  soccer  practice  at  the 
same  time  in  two  different  locations. 

4  My  kids  aren’t  old  enough  to  have  activities  like  soccer.  For  me,  it’s  more  just  the 

difficulty  of  being  the  only  parent  at  home.  When  one  of  them  gets  sick,  they  can’t  go 
to  childcare  and  I’m  the  only  one  that  can  take  care  of  him,  so  I  have  to  miss  work. 

M  So  I’ve  heard  [3  and  4]  say  that  they’re  concerned  about  the  effect  of  deployment  on 
their  children,  and  a  couple  of  you  are  having  difSculty  with  household  responsibility 
issues,  especially  chauffeuring  the  kids  to  activities  and  handling  childcare.  Are  there 
other  problems? 

1  Our  military  pay  has  been  messed  up.  We  didn’t  get  paid  for  the  first  two  months,  and 
now  the  amount  doesn’t  seem  correct. 

2  That  must  be  really  tough.  When  it  does  get  fixed,  though,  you  should  be  happy.  The 
extra  pay  for  my  husband  being  deployed  is  really  nice. 

3, 5, 6, 7,8  [Agree  that  the  extra  pay  is  helpful.] 
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Excerpt  from  a  Focus  Group  of  8  Spouses  of  Deployed  Reservists — Coding 
Positive  and  Negative  Responses 


M  We’ve  been  talking  about  the  positive  aspects  of  deployment,  what  are  the  negative 
aspects  of  having  your  service  member  deployed?  i 

_ _ j  2.3  POSITIVE 

2  Well,  obviously  we  miss  him  when  he’s  gone,  but  we’re  doing  fine,  and  the  additional  [1 
pay  is  really  nice. l 


^n^Mnceme^bounio^miud^h^id^iis^iin^^ie^on^^ai^bouH^nudi^u^ 

my  younger  son  has  been  acting  up  in  school  since  his  dad  left.  |  2  4  NEGATIV^ 


4  I’m  concerned  about  my  boys  as  well.  They’re  not  old  enough  to  understand  why  he’s 
gone,  but  they’re  upset  that  he’s  not  there. 

8  My  kids  are  teenagers,  and  so  they’re  old  enough  to  understand.  For  our  house,  it’s 
more  an  issue  of  logistics.  It’s  just  really  hard  for  me  to  get  my  three  kids  to  three 
different  sets  of  activities  when  I’m  the  only  driver.  Their  friends’  parents  help,  but 
sometimes  my  kids  can’t  do  all  the  things  they  want  to  do  because  I  simply  can’t  get 
them  there. 

1  That’s  the  problem  in  our  family  as  well.  My  daughters  have  soccer  practice  at  the 
same  time  in  two  different  locations. 

4  My  kids  aren’t  old  enough  to  have  activities  like  soccer.  For  me,  it’s  more  just  the 

difficulty  of  being  the  only  parent  at  home.  When  one  of  them  gets  sick,  they  can’t  go 
to  childcare  and  I’m  the  only  one  that  can  take  care  of  him,  so  I  have  to  miss  work. 


M  So  I’ve  heard  [3  and  4]  say  that  they’re  concerned  about  the  effect  of  deployment  on 
their  children,  and  a  couple  of  you  are  having  difficulty  with  household  responsibility 
issues,  especially  chauffeuring  the  kids  to  activities  and  handling  childcare.  Are  there 
other  problems? 


1  Our  military  pay  has  been  messed  up.  We  didn’t  get  paid  for  the  first  two  months,  and 
now  the  amount  doesn’t  seem  correct. 


2  That  must  be  really  tough.  When  it  does  get  fixed,  though,  you  should  be  happy.  The 
extra  pay  for  my  husband  being  deployed  is  really  nice. 

3, 5, 6,7, 8  [Agree  that  the  extra  pay  is  helpful.] 
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Excerpt  from  a  Focus  Group  of  8  Spouses  of  Deployed  Reservists — Coding 
Subthemes 


M  We’ve  been  talking  about  the  positive  aspects  of  deployment,  what  are  the  negative 

aspects  of  having  your  service  member  deployed?  _ 

2.3.1  FINANCIAL 

2  Well,  obviously  we  miss  him  when  he’s  gone,  but  we’re  doing  fine,  and  (POSITIVE) 
additional  pay  is  really  nice. 

3  I’m  concerned  about  how  much  the  kids  miss  him.  They  don’t  talk  abo 
but  my  younger  son  has  been  acting  up  in  school  since  his  dad  left. 

4  I’m  concerned  about  my  boys  as  well.  They’re  not  old  enough  to  understand  why 
he’s  gone,  but  they’re  upset  that  he’s  not  there. 


in-  

2.4.1  CHILDREN’S 
ISSUES 


My  kids  are  teenagers,  and  so  they’re  old  enough  to  understand.  For  our  house,  it’s 

more  an  issue  of  logistics.  It’s  just  really  hard  for  me  to  get  my  three  kids  to  three 

different  sets  of  activities  when  I’m  the  only  driver.  Their  friends’  parents  help,  but 

sometimes  my  kids  can’t  do  all  the  things  they  want  to  do  because  I  simply  can’t  get 

them  there.  2.4.2  HOUSEHOLD 

. .  . .  RESPONSIBILITIES 

That’s  the  problem  in  our  family  as  well.  My  daughters  have  soccer  practice  at  tne 

same  time  in  two  different  locations. 


My  kids  aren’t  old  enough  to  have  activities  like  soccer.  For  me,  it’s  more  just  the 
difficulty  of  being  the  only  parent  at  home.  When  one  of  them  gets  sick,  they  can’t 
go  to  childcare  and  I’m  the  only  one  that  can  take  care  of  him,  so  I  have  to  miss 
work. 


M  So  I’ve  heard  [3  and  4]  say  that  they’re  concerned  about  the  effect  of  deployment  on 
their  children,  and  a  couple  of  you  are  having  difficulty  with  household  responsibility 
issues,  especially  chauffeuring  the  kids  to  activities  and  handling  childcare.  Are  there 
other  problems?  _ 


2.4.3  FINANCIAL 

1  Our  military  pay  has  been  messed  up.  We  didn’t  get  paid  for  the  first  tv 

(NEGATIVEl 

and  now  the  amount  doesn  t  seem  correct. 

That  must  be  really  tough.  When  it  does  get  fixed,  though,  you  should  be  happy. 
The  extra  pay  for  my  husband  being  deployed  is  really  nice. 


3,5,6,7,8 


[Agree  that  the  extra  pay  is  helpful.] 


2.3.1  FINANCIAL 
(POSITIVE) 
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